RapidHarvest Plugings Development

Now you can develop your plugins for downloading files from special sites.
You need to have some knowledge in programming and JavaScript. RapidHarvest gives the possibility to control the download process using standard java script and some provided RapidHarvest functions that you are able to call using JavaScript.

Engines description:

One engine describes one server type. When RapidHarvest starts it searchs for all the files with .eng extension located in the "/engines" directory under the directory where RapidHarvest.exe file is located.

Each .eng file is in XML format and describes one engine.

Engine file description:

<parameters>

<parameter name="engine_name">EngineName</parameter>
<parameter name="url">server.com</parameter>
<parameter name="script">engine.js</parameter>
<parameter name="user_name">username</parameter>
<parameter name="password">password</parameter>

</parameters>

EngineName : Is the Name given to a Engine. It will identify the engine internally so it must be different for each engine.
server.com: It must be replaced by the server that you want to associate to this engine.
engine.js: It points to the JavaScript file that will be used to process the urls associated with this engine.
username: Optional parameter, this value will be accesible from the script using the "user_name" internal script parameter value. Please read the next section for information about internal script parameters.
password: Optional parameter, this value will be accesible from the script using the "password" internal script parameter value. Please read the next section for information about internal script parameters.

Script File Description:

Each engine has associated an script file to it. This is donne using the field "script" in the engine file.
This script file is a file with one mandatory function:
- onStart() : Is the main enter point. This function is called by RapidHarvest each time an URL start being processed.

The two files must be placed under the "engines" directory located in the same directory that rapidharvest.exe.

Communicate with RapidHarvest from Script functions:

For getting function parameters, setting function results and calling RapidHarvest internal functions we use two main Functions:
RH_SetParameter(parameterName,paramaterValue): This function is used for setting one parameter in RapidHarvest.

RH_GetParameter(parameterName) : This function is used for getting one parameter from RapidHarvest.

parameterName : Name of the parameter. You can create your own parameters calling this function. All this paramaters will be valid pending all the time life of the URL download work.
There are special parameter names that will serve to "put" and "get" RapidHarvest information. All this special parameters will be discussed in the next section and depends of the function being called.

Calling RapidHarvest Functions:

RapidHarvest allow calling the following functions from a Script:

post,get,wait,download

All these functions call are asynchronous is why we must define the call back function fore each functionc all. Another constraint is that you can call only one rapidharvest function each time this function is really called when the actual java script function finished.

To call this functions from an script you must set the parameter: "call" using the RH_SetParameter function. A typical example:

function onStart()
{

RH_SetParameter("REQUEST_URL","http://servertopost/action.php");
RH_SetParameter("REQUEST_POST_DATA","postvariable1=value&postvaraible2");
RH_SetParameter("call","post");
RH_SetParameter("callback","onProcessHtmlPageLink();");

}

Explanation:

The first two lines set the parameter "REQUEST_URL" and "REQUEST_POST_DATA" needed by the function to call. The call to RH_SetParameter("call","post") will actually indicate to RapidHarvest the function to call. As you can only call one RapidHarvest function by time then you need to set the variable "callback" to the name of the function that will be called once the RapidHarvest function will finish. You also must define this function in your script. Returning values will be obtained in the CallBack function that you must define in your script. From this function you can call again another function setting the "call" parameter.

Functions Parameters and Results

Each time RapidHarvest call your callback function it returns the result of the called function. You can get this values calling RH_GetParameter(parameterName)

parameterName: is the name of the parameter that we want to get.

Example: From the last example we can define the callback function like this:

function onProcessHtmlPageLink()
{
var bErrors;
bErrors = false;

var sTextHTML,sHTMLHeaders;


sHTMLHeaders = RH_GetParameter("RESULT_HEADERS");
sTextHTML = RH_GetParameter("RESULT_TEXT");

RH_Alert(sHTMLHeaders);

...

}

Explanation: The first Call gets the "RESULT_HEADERS" parameter. In this example this value contains the headers from the last "post" request. It also gets the "RESULT_TEXT" this value contains the HTML text obtained from the last "post" request. The call to RH_Alert(sHTMLHeaders) shows an alert with the headers information. (Please see below for a complet list of functions with parameter and result descriptions)

onStart() function:

This function is the first function called when a download process starts.
You can get the following initial parameters:
"URL" : Original url for the download work.
"user_name" : user name used for this engine. It is set in the engine file.
"password" : password used for this engine. It is set in the engine file.

Remember: you get all this parameters using RH_GetParameter function.

Example: We are going to complete our first example function.

function onStart()
{
sURL = RH_GetParameter("URL");

RH_SetParameter("REQUEST_URL",sURL);
RH_SetParameter("REQUEST_POST_DATA","postvariable1=value&postvaraible2");
RH_SetParameter("call","post");
RH_SetParameter("callback","onProcessHtmlPageLink();");

}

RapidHarvest Functions:

Every function can also set the variables: "ERRORS" , "ERROR_DESC" and "LIMIT_REACHED" to indicate that an error has occurred.

ERRORS: Indicates that an error has occurred.
ERROR_DESC: Gives the description of the error. It will be showed int the column status in the download list.
LIMIT_REACHED: Set the ERRORS variable to "1" and "LIMIT_REACHED" variable to "1" to engage the waiting process for all the downloads that correspond with this engine.

There are also some function utils functions that are not asynchronous:

• RH_Alert(text) : this function shows the content of the text variable.
• RH_HTMLInput(reserved, sInHTML,sTemplatePath): This function will show a dialog using the specified HTML template
  • reserved : must be allways 1.
  • sInHTML : a list of variables to reeplace in the template file. this variables has the form of: variable1=urlencoded1&variable2=urlencoded2....
    The HTML template file must define HTML <div> sections that corresponds to these variables.
  • sTemplatePath: Path absolute or relative to the HTML template file.
    
  
  Example:
  
  var sInputText = escape('<input id="field1" name="field1" size="4" maxlength="3" type="text">');
  var sInHTML = "div1=Hello&div2="+sInputText;
  RH_HTMLInput(1,sInHTML, "/engines/inputtemplate.~html");
  
  inputtemplate.~html content:
  

  <BODY> <div id="div1"></div> <div id="div2"></div> </BODY>

  
  This will generate an HTML Dialog with the following source:
  

  <BODY> <div id="div1">Hello</div> <div id="div2"><input id="field1" name="field1" size="4" maxlength="3" type="text"></div> </BODY>

  
  As you see, the content of the two <div> sections has been replaced by the specified parameters in the sInHTML variable.
  After calling this function you can get the input from the user using a call like this:
  
  var sField = RH_GetParameter("field1");
  
  As you see you can get all the values typed in the input elements of type "text" that you difine in your HTML template.
  

Extra Useful Parameters

You can use also the RH_GetParameter function to obtain some useful values:

• RH_GetParameter("windows_temp_directory"): obtains the windows temp directory path.
• RH_GetParameter("user_name"): obtains the user name definde in the engine .eng file .
• RH_GetParameter("password"): obtains the password defined in the engine .eng file.
  
  MORE WILL COME in the future.