Task Management - for Developers

From Sense/Net Wiki
Jump to: navigation, search
  •  
  •  
  •  
  •  
  • 100%
  • 6.5
  • Enterprise
  • Community
  • Planned

Overview

The Task Management component is a standalone application that offers an efficient and scalable task management for Sense/Net ECM (or any other 3rd party client application). This article is for developers about extending the task management framework, e.g. creating custom task executors.

Before reading this article, please take a look at the main Task Management article for information about the components described here.

Please note that this feature is only available in the Enterprise edition.


Deployment

About deploying the Task Management application and task executor tools, please visit the following article:


TaskManagement Core library

The Task Management API offers a core library that you can reference in your project and take advantage of the helper classes and interfaces there:

  • SenseNet.TaskManagement.Core.dll

This library is installed with the task management web application, and can be found in the bin folder. Every component of this system uses this library as it contains (among other things) the common classes for communication.

It is possible to connect to the task management component without the library above, but it provides an easier way for communication. Sense/Net ECM already references this library so you can find it in our web\bin folder too. If you are developing for Sense/Net ECM and want to register your custom tasks, please do it through the API of this library.

The following list contains the main classes and interfaces in the core library. Use it as a reference when reading the sections below.

  • ILogger and LoggerBase: a basic logging mechanism with a built-in implementation that sends messages to the EventLog. See usage examples in the agent tool.
  • ITaskManager: a simple interface for developers who want to connect to the task management web app from a 3rd party application instead of Sense/Net ECM. It describes the methods that need to be implemented in case of a client.
  • RegisterApplicationRequest: defines the properties that need to be provided when a client application registers itself.
  • RegisterTaskRequest: defines the properties that need to be provided when a task is registered.
  • RegisterTaskResult: a result object containing the actual task that was registered successfully.
  • RepositoryClient: this class provides the static API for communicating with the Task Management web app. It hides the REST API calls from the client.
  • SnTask: represents a registered task.
  • SnSubtask: contains properties and helper methods for custom executors to be able to provide task events and progress info.
  • SnTaskResult: the main object received by finalizers.
  • SnTaskError: finalizers will receive this object in case of an error occured during task execution.
  • SnTaskEvent, SnHealthRecord, SnProgressRecord: objects sent to task monitor clients through the monitor hub to provide information about task execution.
  • TaskManagementException: the exception type that wraps all kinds of errors in the communication between the client application and the task management center.

Custom task executor

In this section we describe how can you plug in your custom task executors into the task execution pipeline.

Starting a new task

You are able to start your custom task (or even one of the built-in tasks of Sense/Net ECM) from your code using the Task Management API. In the core product when a user uploads a document, the preview provider registers a new task to generate the first couple of preview images.

var requestData = new RegisterTaskRequest
{
    Type = taskType,			// task type, e.g. "PreviewGenerator". This is what determines the executor tool to be started by the agent (--> PreviewGenerator.exe).
    Title = taskTitle,			// title, will be displayed on the monitor UI
    Priority = priority,			// determines the order of executing task jobs
    Tag = tag,				// an optional category, used by the monitor UI
    AppId = appId,			// unique application identifier
    TaskData = previewData,		// task-specific data; a serialized JSON object is recommended. It will be passed to the executor without modification
    FinalizeUrl = finelizeUrl		// optional task-specific finalizer URL (see example below)
};
 
TaskManager.RegisterTask(requestData);

The example above uses the Sense/Net ECM static API for registering the task, you dot not have to worry about sending REST API requests to the Task Management web app.

Finalizing tasks

It is possible to execute custom code when a task is finished (even if you do not write a custom executor). For example you may want to log errors or perform some task-specific business logic at the end. Developers may inject their custom code even into the finalize process of built-in tasks. The finalizer executes in the context of the application that registered the task (in our case a Sense/Net ECM instance) and receives all the information about the task and the execution result.

About creating a finalizer in a 3rd party application, please visit the #3rd party applications section later in this article.

The finalizer is actually a simple REST API endpoint that the Task Management web app calls when a task executor finishes its work. This is how the task management web app tries to construct the finalizer URL:

  • if a URL was provided during task registration, it will use that
  • if not, the global finalizer URL will be used (provided during app registration)
  • if the URL is relative, it will be prefixed with the base application URL (provided during app registration)

In Sense/Net ECM it is advisable to use an OData action as a finalizer that receives an SnTaskResult object as a parameter. Please visit the following articles on how to do that:

The received SnTaskResult object contains the following values:

  • MachineName: the name of the machine that the task was executed on
  • AgentName: the unique name of the agent that executed the task.
  • Task: the task object that was registered originally. It contains the full task data provided the code that registered the task.
  • ResultCode: the result code of the executor tool.
  • ResultData: optional additional information written to the Console by the task executor with the prefix ResultData:.
  • Error: an error object containing any errors that occurred during task execution and were written to the Console by the task executor with the prefix ERROR:.


It is advisable to include a call to the built-in generic finalizer method even in your custom finalizer code. This way we will take care of logging, and you will only have to take care of your custom business logic. Use the following line at the beginning of your finalizer to do that:

TaskManager.OnTaskFinished(result);

Finalizing a custom task

If you created your own custom task, you can create a finalizer for it and provide its URL during task registration. Please visit the articles above on how to create a custom OData action for this.

Finalizing a built-in task

Built-in tasks usually have their own finalizer API endpoint as an OData action. This means you will find a generic OData application in the Content Repository for them. For example in case of the preview generator, this is the following content:

  • /Root/(apps)/GenericContent/DocumentPreviewFinalizer

If you edit the application content, you can change the method that actually handles the action. This way you may customize the behaviour even of a built-in finalizer. Please visit the articles listed above for details about OData actions.

Creating a task executor

It is possible to create your own task executor for your business requirements and use the Task Management framework to execute them. You have to create a new console application in Visual Studio. You have to handle the communication with the agent in your tool:

  • parse the incoming parameters at the beginning of your Main method
  • regularly provide the progress status of the execution for the agent and the monitor GUI to know that the tool is still alive

The name of the executor tool is important: agents will find and start executors based on the name of the registered task. If you registered a task with the name FolderCompress, than your executor tool should be in the FolderCompress folder under the TaskExecutors folder on agent machines and it should be named FolderCompress.exe.

You may even have multiple different versions of the same executor if you name them differently and register tasks with the appropriate names (e.g. FolderCompress and FolderCompressV2). This is useful in case of the same Task Management framework serves multiple different versions of Sense/Net ECM or other client applications.

Incoming parameters

The agent starts the task executor process and provides the necessary parameters as command line arguments in the following format:

MyTaskExecutorTool.exe <ParamName1>:<Value1> <ParamName2>:<Value2>

The parameters are the following:

  • USERNAME: username for the client app (usually a Sense/Net ECM instance).
  • PASSWORD: password for the user above.
  • DATA: all custom data that was provided when the task was registered (e.g. the portal url or a content id). This is usually a JSON formatted string that can be deserialized easily - see an example in the Starting a new task section.

Progress info and subtasks

A single task (e.g. generating preview images for a document) may consist of one or more sub tasks. For example a subtask may be to download a document from the portal, then another subtask is to generate the images. The point of subtasks is to be able to provide meaningful information for the administrator who monitors the execution of tasks.

The subtask API lets developers mark the start and end of subtasks and provide numerical progress information for the monitor. The information you write into subtasks will go directly to the user interface (and will be displayed as titles and progress bars). Main events, like start and finish will also be persisted to a database for later use.

A best practice is to decompose your task into subtasks and provide progress messages regularly - for example after generating every preview image. This is why this depends on the nature of the task.

The subtask API lets you send messages to the agent. Under the hood it basically writes to the Console, which you can also do on your own if you want to. Please refer to the source code for more details.

This is a simplified example for using subtasks, without real error handling.

var downloadingSubtask = new SnSubtask("Downloading", "Downloading file and other information");
downloadingSubtask.Start();
 
try
{
    var fileInfo = GetFileInfo();
 
    // write some progress
    downloadingSubtask.Progress(10, 100, 2, 110, "File info downloaded.");
}
catch (Exception ex)
{
    //Logger.WriteError
}
 
using (var docStream = GetBinary())
{
    if (docStream == null)
    {
        downloadingSubtask.Finish();
        return;
    }
 
    downloadingSubtask.Progress(100, 100, 10, 110, "File downloaded.");
    downloadingSubtask.Finish();
 
    _generatingPreviewSubtask = new SnSubtask("Generating images");
    _generatingPreviewSubtask.Start();
 
    // ...generate preview images...
 
    _generatingPreviewSubtask.Finish();
}

Example

The following code snippet demonstrates the way you are able to create a custom task executor tool as a console application.

using System;
using System.IO;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
 
namespace MyTaskExecutor
{
    class Program
    {
        static void Main(string[] args)
        {
            if (!ParseParameters(args))
            {
                //Logger.WriteWarning("Task process arguments are not correct.");
                return;
            }
 
            try
            {
                ExecuteTask();
            }
            catch (Exception ex)
            {
                //Logger.WriteError(ex);
 
                // if logger writes a standard error information to the console,
                // the task finalizer receives an SnTaskError instance with the
                // type, message and stack trace of the catched exception.
                // The simplest way of writing the exception is the following:
                //Console.WriteLine("ERROR:" + SnTaskError.Create(ex).ToString());
            }
        }
 
        private static void ExecuteTask()
        {
            //TODO: implement task logic, create subtasks and write progress
            var subTask = new SnSubtask("Working...");
 
            for (var i = 0; i < 10; i++)
            {
                //do stuff...
                //write progress...
                subTask.Progress(progress, 100, overall, max);
            }
 
            //finish last subtask
            subTask.Finish();
        }
 
        private static bool ParseParameters(string[] args)
        {
            foreach (var arg in args)
            {
                if (arg.StartsWith("USERNAME:", StringComparison.OrdinalIgnoreCase))
                {
                    //TODO: store Username
                }
                else if (arg.StartsWith("PASSWORD:", StringComparison.OrdinalIgnoreCase))
                {
                    //TODO: store Password
                }
                else if (arg.StartsWith("DATA:", StringComparison.OrdinalIgnoreCase))
                {
                    var data = GetParameterValue(arg).Replace("\"\"", "\"");
 
                    var settings = new JsonSerializerSettings { DateFormatHandling = DateFormatHandling.IsoDateFormat };
                    var serializer = JsonSerializer.Create(settings);
                    var jreader = new JsonTextReader(new StringReader(data));
                    dynamic taskData = serializer.Deserialize(jreader) as JObject;
 
                    //TODO: get values from taskData
                }
            }
 
            return true;
        }
 
        private static string GetParameterValue(string arg)
        {
            return arg.Substring(arg.IndexOf(":", StringComparison.Ordinal) + 1).TrimStart(new[] { '\'', '"' }).TrimEnd(new[] { '\'', '"' });
        }
    }
}

Error handling

When an unexpected event happens in an executor tool, the system should know about it. This includes the Task Management central application and also the monitoring UI. We provide a simple API for providing detailed error messages in the executor. The following example demonstrates how can you send error messages to the system. This will write a formatted message to the console that is monitored by the agent and the message will be parsed and sent to the Finalizer on the portal.

Note that the parameter below should be an object that is executor-specific. In this case it contains information about preview image generation.

// note the ERROR: prefix
Console.WriteLine("ERROR:" + SnTaskError.Create(ex, new
{
    ContentId = contentId, 
    Page = page, 
    StartIndex = startIndex, 
    Version = version,
    Message = message
}));

Deploying the task executor

To let agents use your task executor, you need to create a separate folder for it under the TaskExecutors folder on every agent machine. The name of the folder should be the name of your task (which is the same as the name of the tool, except the .exe extension - in the previous example this would have been MyTaskExecutor) and copy all the necessary libraries and config files there needed by the tool. The agents will automatically discover and execute your tool when a new task arrives for it (a service restart may be needed). For details please visit this article:

Custom task manager

It is possible to customize the behavior of the task management framework by creating a custom task manager class inside Sense/Net ECM. You may inherit from the built-in DefaultTaskManager class or create your own implementation of the ITaskManager interface. Currently you have to override/implement the following methods:

  • OnTaskFinished: called when a task execution was finished by the task management component.
  • RegisterApplicationAsync: called during app start.
  • RegisterTaskAsync: called when somebody wants to register a task.

About registering your custom task manager class please see the web.config details here.


3rd party applications

In this section you can learn about connecting to the Task Management component from a 3rd party application (instead of Sense/Net ECM). Sense/Net ECM developers can simply skip this section as we already integrated the component into the product. It is advisable to add a reference to your project for the SenseNet.TaskManagement.Core library mentioned above to make the integration super easy.

Registering your application

All applications that want to send tasks to the Task Management app need to register themselves first. In case of a web application this may happen in the App_Start method. The registration is a simple call to the RegisterApplication method in the core library mentioned above.

var requestData = new RegisterApplicationRequest
{
    AppId = myAppIdFromConfig,
    ApplicationUrl = myUrl,
    TaskFinalizeUrl = finalizeBaseUrl
};
 
await RepositoryClient.RegisterApplicationAsync(taskManagementUrl, requestData);

This will be executed every time the app starts so that the Task Management center will know that this is a real and live application. The request object has the following options (see the details in the deployment article on how we store and use these values):

  • AppId: the unique identifier of your application.
  • ApplicationUrl: base url for all callbacks.
  • TaskFinalizeUrl: optional global url for a central finalizer API entry point. In Sense/Net ECM this is not used, as we use a document-specific OData action for finalizing our preview generator tasks; and that url is provided at every task registration.
  • AuthenticationUrl: (not used yet)
  • AuthorizationUrl: (not used yet)

Alternatively you may post the object above (serialized in JSON format) to the following API URL in the Task Management web application:

Registering a task

This works the same way as the #Starting a new task example above, although you will need to use the TaskManagement core method instead:

await RepositoryClient.RegisterTaskAsync(taskManagementUrl, requestData);

Alternatively you may post the request object above (serialized in JSON format) to the following API url in the Task Managemebt web application:

Error handling

It is possible that the task registration fails. This can be because of a network error, or something is not right on the Task Management web app's side. In this case you need to make sure that your operators get notified, because registering a task is an essential part of this system. A special case of a registration error is when the Task Management system says that it does not know about the application id you provided. This is usually caused by the fact that the Task Management web app was inaccessible when your application started, thus app registration failed. In this case the RegisterTaskAsync method will throw a TaskManagementException with the following message:

  • UnknownAppId

In this case it is your responsibility to register your application again, then retry the task registration immediately.

Please note that the above applies only for 3rd party applications. Sense/Net ECM handles this for you.

Creating a finalizer

As a finalizer has to be a REST API endpoint, you can create one in your application with the technology of your choice. The only constraint is that it needs to be able to receive an HTTP request containing an SnTaskResult object in JSON format. Please see the details of this object above.

Authentication and authorization

Coming soon.

Custom Task Monitor GUI

The Task Monitor user interface in Sense/Net ECM is built using Kendo UI controls and a SignalR connection to the Task Management web application (through a hub in Sense/Net ECM). This means you may build a custom UI for your application, if you make use of the same SignalR hub methods and handle the same events.

Built-in monitor plugin

Sense/Net ECM contains a plugin that we use to display task events and monitor execution progress. You can fork this plugin in your application if you want a similar look and feel. The plugin can be found in the Content Repository:

  • /Root/Global/scripts/sn/SN.TaskMonitor.js

The plugin needs an options parameter to be able to work correctly. You can initialize it the following way:

$(document).ready(function () {
    $("#taskDataGrid").SenseNetTaskGrid({
        appId: appId
    });
});

appId is the same predefined appId that your application is identified by on the Task Management web.

For details about the necessary UI elements (e.g. html tags) please refer to the source code.

Brand new GUI

It is also possible to built a brand new user interface from the ground up. You will need to understand how ASP.NET SignalR works to be able to connect to the server.

In this section we list all the methods available in the server-side task monitor hub, and all the client methods that should be implemented by you to handle the messages sent by the server.

Task Monitor hub

From your client you can connect to the task monitor hub in the task manager web application using the following code:

var connection = $.hubConnection(taskManagementUrl);
connection.qs = { 'appid': appId };
 
var taskMonitorHubProxy = connection.createHubProxy('taskMonitorHub');
 
// TODO: define client-side methods here, called by the server
 
connection.start().done(function () {
    console.log('Connected successfully to TaskManagement server.');
    //TODO: define startup logic (initialize UI)
});

Through the hub initialized above you may call a couple of server-side methods the following way:

taskMonitorHubProxy.invoke("GetUnfinishedTasks", appId, null)
	.done(function (result) {
		for (var i = 0; i < result.length; i++) {
			processTaskEvent(result[i]);
		}
	})

The following methods are available for the client to call:

  • GetUnfinishedTasks: Loads all tasks from the database that are registered, but not finished or failed. Parameters:
    • appId
    • tag (can be null)
  • GetDetailedTaskEvents: Loads all task and subtask events for a single task. Parameters:
    • appId
    • task id
    • tag (can be null)

Client methods

Before connecting to the server, you will need to define the methods on the client side that will be called by the server. These are the methods that you should implement in your custom UI.

// called when a task or subtask starts or ends
taskMonitorHubProxy.on('onTaskEvent', function (taskEvent) {
    processTaskEvent(taskEvent);
});
 
// called periodically to ensure the client that the agent is still alive
taskMonitorHubProxy.on('heartbeat', function (agentName, healthRecord) {
    processHeartbeat(agentName, healthRecord);
});
 
// called when any of the executors provide some progress information
taskMonitorHubProxy.on('writeProgress', function (progressRecord) {
    processProgress(progressRecord);
});

Related links


References