Generating the Transactions and Proceedings of the Royal Society of New Zealand website, using XSLT running on a Condor grid

Abstract

This is a report on an experimental NZETC project to speed up the automated regeneration of a large website by using VUW's Condor computing grid. The aim of the project was to investigate the feasibility of using the grid as a platform for website generation. The project did not extend to generating the entire website, though it did produce the bulk of it, and did successfully demonstrate the feasibility of using the grid for similar work in future. The grid-based application has shown the grid to be a good platform for generating large websites from XML source materials. The application was able to generate around 80 thousand web pages in about 4 hours, making it many times quicker than a system based on a single computer. The project also highlighted a few technical issues with the Condor grid, relating to the use of file folders in grid jobs; prioritisation of grid nodes based on the opening hours of the computer labs; and the special configuration of computers running Microsoft Windows.