Sun June 6th 2010 -- Version 1.25 --------------------------------- - Added this ChangeLog.txt file. - Added new scheduler into project server code. A new table event_queue has been added in the database. - The resource daemon has been modified to add a resource hash into the job_specifics tag when a job is accepted. It is used by the daemon to check if a job has been taken over by another daemon on the same or different resource and should ignore the job but finish or abort gracefully on the resource if possible. - Added a deamon_pulse into the job_queue to detect daemons not running as they should (i.e. on a grid). Jobs can now be requeued. - Added new priority field into the job_queue table. This allows a future version of the scheduler to supply QOS by assigning a different priority to jobs. Currently the priority is set equal to the time stamp of submit. Sat June 26 2010 -- version 1.26 -------------------------------- - Added python client class into python dir. - Moved daemon/src into root, adjusted makefile and so on. Wed July 14 2010 -- version 1.27 -------------------------------- - Adapted daemon and tools to use persistent connections by reusing the cURL handle. - Configured Apache for performance using keep-alive options. - Updated documentation. - Made sure that the python interface class also handles the persistent / keep-alive connections correctly. - Made sure aborting jobs without a pulse are also nicely taken care of. - Studied LGI under load and updated statistics in the documentation. - Made sure that MySQL connections are nicely closed on exits. - Found a small race-condition in the resource request work API, it has been fixed and the resource locking has been optimized. - Made sure cURL is setup and cleaned up nicely. - Removed another small race-condition for resources and removed table locks from algorithm in interface job locks. - Using ctype_digit for checks and check limit values for queries. - Optimized locking for resources a bit more. - Removed small race-condition in daemon_pulse field. - Tweaked MySQL setup for better performance, ran tests again for updated stats in LGI.pdf. - Added some extra indices to job_queue to avoid MySQL sorting a bit more. The number of slow_queries dropped significantly. - Removed duplicate indices. - Use explicitly specified index for work requests and job lists so no 'ORDER BY' is needed to sort. Updated stats and documentation. Big improvements gained. - Removed some php error messages for repositories. - Added a check_running script for the LGI scheduler. Thu August 19 2010 -- version 1.28 ---------------------------------- - Removed some old documentation and added new. - Allow for project server scheduler unlimited run time. - Limited the number of work request a resource can do. - Made index UNIQUE in running_locks. - Made sure interfaces job list requests do sort the jobs on state_time_stamp by using the available index, even when the table is small and the MySQL optimizer likes to ignore the indexs we made. - Made sure that for the basic_interface the SID (to counter XSS attacks) is not reset each time index.php is hit. Fri November 19 2010 -- version 1.29 ------------------------------------ - Patched Ssl.inc to work with newer PHP 5.x (patch of Willem v. Engen). - Added 'cd' trick to some tool scripts so they can be run from any other directory. - Added extra logging to stderr option for qstat tool. - Made sure that for CLI tools arguments are numeric if needed. - Added job count feature to request work for resources for future pilot job frameworks. - Removed small update bug for master-slave servers. - Added slave server capabilities report into daemon. Wed August 31 2011 -- version 1.30 ---------------------------------- - Patched repository content listing code. It had a security hole found by Willem v. Engen. - Removed some dead code in interface job state API. - Added support for retreiving job count in interface API. - Added support for specifying owners allowed or denied for resources in resource request work API. - Removed small bug generating php error messages in Utils.inc. - Added support for reporting allowed and denied owners in daemon when requesting work. - Added SIGHUP support in daemon for restarting logger. - Made sure server updates use keep alives and caching in posting. - Added support for relative paths in daemon config file, thanks to Willem v. Engen. - Added the .spec files in specs to build RPMs on RHEL systems. - Added support for more than 32k jobs in repository by adding extra fan-out directory based on unique name. Limit is now 134M jobs on ext3 filesystem. - Made sure .spec files create upgradable rpms and use correct source URL. - Removed small issues with relative paths in daemon config file. - Made sure paths for daemon can have spaces. - Updated default scripts a bit. - Removed small scheduling bug from daemon. - Tested .specs and code against RHEL 6; removed two small curl issues. - Made sure that LGI python code can deal with limit=0 values. - Updated documentation. - Added configuration reload capabilities on SIGHUP into daemon. - Added logic to reset libcurl handle; some versions of libcurl seem to stop working after a fews days without any apparent reason. - Removed small bug from escape string causing check system limits scripts to fail. - Removed small lock_state issue. - Removed possible XML injection issues found by Willem v. Engen which caused wrong output in parsing response XML tags. Mon February 23d 2015 -- version 1.31.9 --------------------------------------- - Added LGI server version number. Set initial version to 1.31. - Added API version number. Set initial version to 1.31. - Made XML tests more rigorous, added escaping and updated documentation. - Added weak server certificate hostname check option into cli and daemon. - Revised the repository listing approach. - Updated documentation regarding certificates a bit. - Made sure code also compiles on OS X; implemented BaseName. - Made sure xml now allows for "" comments. - Made sure job specifics in LGI.py can also be supplied. - cURL handles are closed now after scheduling cycles to avoid server side FIN_WAIT2 states. - Imported the .spec files from willem v engen and his build script. - Switched to x.y.z versioning system. Started with 1.31.1. - Made sure daemon requests work from applications starting with a random application each server request. - Added privatekeys directory and adapted .spec files to save configs when updating. - Made sure daemon can handle big files if compiled in 32 bit. - Added option to compile daemon not using double fork daemonizing of run scripts. Add __NO_DOUBLE_FORK__ option into defines to enable. This was needed to run the daemon on the nikhef grid nodes. - Fixed a small race condition in spawning the job run scripts while still writing the pid of child. - Made sure the start and stop scripts created by .spec files actually start and stop at boot and shutdown. - Made sure the default reset of running jobs without a pulse does not alter the target_resources field anymore. - Made sure the project to start requesting work from is picked at random too. The project server of the project is not chosen at random to allow for static load ballance over servers. - Made sure ManageDB runs all updates and regenerates resource acl file for repositories. - Made sure libCurl now verifies hostname of cert of project server correctly by default and can handle mod_deflate decompression. - Added small tweak into LGI project server scheduler to counter users submitting large blocks of jobs and claiming the queue. - Updated documentation and .spec files for RHEL 5, 6 and 7. - Made the certificate file namings for the .rpm based installs easier. Mon Dec 23th 2017 -- version 1.32.8 ----------------------------------- - Fixed small xml parsing bug in php code. - Restored requirement that resource capabilities field must be XML in the daemon configuration file. It is not escaped anymore by daemon. Version number updated (also of API). - Made sure the LGI python interface code also works for windows. - Added versioning patching of .spec files in rpm build script. - Added support for default application, default target resources, default read and write access configuration in .LGI for LGI_qsub. - Added option to adjust connection timeout for LGI_filetransfer. - Removed small bug from timeout option code in LGI_filetransfer. - Made sure correct settings are read in from new config dir if arg was given. - Changed default configs to match new urls. - Fixed little PUT and DELETE cgi bug in repository checks. - Fixed fractional file date time output of find in repository content list. - Reset basic interface session id if not defined to get rid of php error in log. - Fixed small bug in fractional time strpos fix. Mon 8th April 2019 -- version 1.33 ---------------------------------- - Cleaned up daemon example scripts. - Added slurm resource hello world application scripts. - Remove el5 rpm build environment. - Fixed bug regarding lingering repositories when submitting jobs fail. Tue 9 July 2019 -- version 1.34 ------------------------------- - Updated some documentation regarding RHEL 7 installs. - Removed rpm spec files and build env for RHEL 6. - Fixed RHEL 7 spec file SELinux attributes. - Updated general LGI documentation. Mon 1 September 2019 -- version 1.35 ------------------------------------ - Made sure LGI works with python 2 and 3. Thu 10 Oktober 2019 -- version 1.36 ----------------------------------- - Removed typo in daemon logging. - Made example script default use python3. - Created spec files and tested RPMs for CentOS 8 for cli, python and resources. LGI_server not supported yet due to mysql drop in php 7. Tue 12 November 2019 -- version 1.37 ------------------------------------ - Created spec file and tested LGI_server rpm for CentOS 8. To port the LGI php server code to PHP 7, a wrapper mysql.php was introduced based on the work of Sjoerd Maessen: https://github.com/e-sites/php-mysql-mysqli-wrapper. - Added default empty values for input and output blobs of job_queue. - Updated documentation for both CentOS / RHEL 7 and 8. Fri 30 April 2021 -- version 1.38 --------------------------------- - Fixed small off by one php bug in Utils.inc. - Found a way to make python3 play nice uploading binary files via job submit in the LGI python interface when files contain data clashing with utf-8 codec now used by default in str type reading text from files. Mon 13 December 2021 -- version 1.39 ------------------------------------ - Fixed a small utf-8 string to hex encoding odd-ness in LGI.py for python3. Wed 20 April 2022 -- version 1.40 --------------------------------- - Fixed lacking default resource_capabilities in ManageDB. Tue 21 June 2022 -- version 1.41 -------------------------------- - Added .spec files for el9. Thu 16 nov 2023 -- version 1.42 ------------------------------- - Tweaked scheduler a bit for longest waiting job priority.