
After I was stuck with designing
an algorithm for automatic layout of random graphs, I shifted over to some other interesting problems at work. On the basis of the excellent book "Lectures on Monte Carlo Methods" by Neal Madras I have implemented two simualted annealing algorithms.
The first algorithm is an application to the backpack problem which is given as follows: There are a number of items, each having a specific weight and value. You are a burgler with a backpack which can only carry a limited amount of weight - of course you want to theft as much valuable items as you can, but not all items fit into your bag. A very simple but quite powerful algorithm is as follows:
1. Calculate the ratio "value per weight" for each item.
2. Sort all items according to this ratio.
3. Beginning with the items with the highest ratio, try to put each item into your bag in the order of that sorted list.
Although this is a very simple and straight-forward solution, it gives very good results (though these do not have to be optimal).
After that I tried to implement an approach using simulated annealing, which works as follows:
1. Empty your bag.
2. Select an item at random.
3. If the item is in your bag, remove with probability exp(-beta ratio log(iteration)).
If the item is not in your bag, and there is still enough room, put it into your bag.
4. goto 3.
The value
beta*log(iteration) represents the inverse temperature at each iteration and is raised logarithmically. Of course simulated annealing is a very general approach, which does not exploit the special structure of a given problem, so you cannot expect very good solutions to a given problem. But when running long enough (about 1000 iterations) I was able to get near the solution given by the simple algorithm in the case of 100 items (each with a uniformly distributed weight and value) and a bag pack which can carry at most a weight of 20.
Another interesting problem is the traveling salesman problem: Given a number of cities with their location, find a tour which visits each city and has minimal length among all such tours. This problem is known to be NP-hard (just like the first one), so probably no efficient algorithm for the opimal solution does exist. The simulated annealing approach looks very similar, here we start at a trivial tour that simply visits all cities in the order of appearance and exchanged two cities on the tour with a probability depending on the win/loss of tour length.
You can download the Maple worksheets here:
Backpack.mw and
Traveling Salesman.
Kaya