X-Git-Url: http://wamblee.org/gitweb/?a=blobdiff_plain;f=crawler%2Fkiss%2Fdocs%2Fcontent%2Fxdocs%2Findex.xml;h=e84efaa60f3642019945e0c2c3ca1c5fdcb506d1;hb=9123cf3af79422d515e8b5a05fd5472653c57dc6;hp=02d1a4668fe6609e68ac1f56835925650c8876ef;hpb=a0108c619a90059c7b550e3e0e520e863c0365df;p=utils
diff --git a/crawler/kiss/docs/content/xdocs/index.xml b/crawler/kiss/docs/content/xdocs/index.xml
index 02d1a466..e84efaa6 100644
--- a/crawler/kiss/docs/content/xdocs/index.xml
+++ b/crawler/kiss/docs/content/xdocs/index.xml
@@ -20,7 +20,57 @@
Automatic Recording for KiSS Hard Disk Recorders
-
+
+
+ KiSS makes regular updates to their site that sometimes require adaptations
+ to the crawler. If it stops working, check out the most recent version here.
+
+
+ Changelog
+
+
+ 7 September 2006
+
+ - KiSS modified the login procedure. It is now working again.
+
+
+
+ 31 August 2006
+
+ - Added windows bat file for running the crawler under windows.
+ Very add-hoc, will be generalized.
+
+
+
+ 24 August 2006
+
+ - The crawler now uses desktop login for crawling. Also, it is much more efficient since
+ it no longer needs to crawl the individual programs. This is because the channel page
+ includes descriptions of programs in javascript popups which can be used by the crawler.
+ The result is a significant reduction of the load on the KiSS EPG site. Also, the delay
+ between requests has been increased to further reduce load on the KiSS EPG site.
+ -
+ The crawler now crawls programs for tomorrow instead of for today.
+
+ -
+ The web based crawler is configured to run only between 7pm and 12pm. It used to run at
+ 5am.
+
+
+
+
+
+ 13-20 August 2006
+
+ There were several changes to the login procedure, requiring modifications to the crawler.
+
+
+ - The crawler now uses the 'Referer' header field correctly at login.
+ - KiSS now uses hidden form fields in their login process which are now also handled correctly by the
+ crawler.
+
+
+
Overview
@@ -66,8 +116,8 @@
The easy way to start is the
- standalone program binary version
- or using the web
+ standalone program binary version
+ or using the web
application.
@@ -238,7 +288,9 @@
The screen should show an overview of the last time it ran (if
it ran before) as well as a button to run the crawler immediately.
Also, the result of the last run can be viewed.
- The crawler will run automatically every morning at 5 AM local time.
+ The crawler will run automatically every morning at 5 AM local time,
+ and will retry at 1 hour intervals in case of failure to retrieve
+ programme information.
@@ -256,11 +308,17 @@
General usage
When the crawler runs, it
- retrieves the programs for today. As a result, it is advisable
+ retrieves the programs for tomorrow. As a result, it is advisable
to run the program at an early point of the day as a scheduled
task (e.g. cron on unix). For the web application this is
preconfigured at 5AM.
+
+ If you deploy the web application today, it will run automatically
+ on the next (!) day. This even holds if you deploy the application
+ before the normal scheduled time.
+
+
Modifying the program to allow it to investigate tomorrow's
programs instead is easy as well but not yet implemented.