Tags / Netkernel
External Request ProfilerWe’ve been working recently with a client, on a solution, revolving around crunching large volumes of data. The scale of data meant that even on a large cluster the computation takes many hours. The usual mode of operation for these compute jobs is to set them off in the evening, and return the next working day to see the results. I was working with them to capture some engineering figures and statistics to assess the performance and opportunities to optimise or re-design.
Generalizing Dynamic Programming
- Using dynamic programming techniques to optimise general compute efficiencyDynamic programming is an approach to efficiently implementing certain classes of algorithms. As a feeble excuse for not noticing this earlier, the term is a little confusing: it isn’t about programming or about doing it dynamically! It was invented in the 1950s by Richard E. Bellman whilst working for RAND Corporation. Apparently he deliberately named it obtusely to avoid his work being considered research! I discovered this term recently when looking into existing research about algorithm optimisation and specifically why the systemic memoisation in NetKernel can optimise solutions to certain problems particularly well.
Travelling Salesman Problem
- Study of TSP running on NetKernel with cache replacing dynamic programmingThis paper offers an analysis of the application of NetKernel and Resource oriented computing to the well known Travelling Salesman Problem (TSP). This discussion ignores the many approximate approaches to solving this problem which are necessary for real-world problems of any significant complexity. The aim, however, is to show how ROC can automatically optimise a solution which it knows nothing about other than empirical data collected during execution. The algorithm structures the problem into sub-problems many of which overlap with each other - this occurrence is called overlapping subproblems.
Practical State Machines in NetKernel
Following up on my previous post, ROC Hockey, where I introduced our new implementation of Hierarchical State Machines on NetKernel I’ve decided to cover the topic of how they are actually used in practice in an ROC system. Since that post we have released the first version of Lang/SCXML urn:com:ten60:netkernel:lang:scxml into the Enterprise Edition repositories. If you want to have a play with it yourself here are the steps: