How to access the blob storage directly

From Sense/Net Wiki
Jump to: navigation, search
  •  
  •  
  •  
  •  
  • 100%
  • 6.5.5
  • Enterprise
  • Community
  • Planned

Overview

SenseNet ECM offers several different options for storing binaries. By default everything is stored in the database, but it is also possible to store binaries in an external blob storage. About the concept and basics on how to configure or create custom blob providers, please visit the main article:

This article is for developers about how to read or write binaries that are stored in an external blob storage directly, without having to send them through the portal's REST API.

Details

The idea is that you want to access the blob storage (e.g. read or write files) but without putting too much pressure on the web servers. In this solution you will still have to send a few requests to the portal, but direct binary access (loading or writing the bits) will go through a local blob storage API. In case of creating a new file the workflow is the following:

  1. Send an initial request to the portal for a token.
  2. Write the binary using the token.
  3. Send a finalize request to the portal.

The start and finish requests are necessary because we still have to write to the metadata storage when working with binaries. This method takes part of the load off the web server by writing the bits directly to the blob storage.

If you use the Client library for these operations (as we recommend) you would not even have to send the requests above manually: it will take a single method call to write a file to the blob storage. See an example below.

Blob storage component usage

Currently these are libraries that you have to reference if you want to work with the blob storage directly in your external tool (for example a console application that imports files).

  • SenseNet.BlobStorage (part of the core product)
  • SenseNet.Common (part of the core product)
  • SenseNet.Client (standalone component)

In case you need transactionality in your tool - for example because you decided to use the Filestream feature of SQL Server or you you have another blob provider that requires this - you will have to inject a transaction factory at the start of your application:

CommonComponents.TransactionFactory = new SqlTransactionFactory();

This is not necessary in most cases, e.g. when you work with the default SQL storage configuration.

Configuration

It is important to know that configuration in the external tool have to be the same as on the portal to make this component work correctly. Please review the necessary config values here:

You will also have to make sure that the usual platform-specific configurations (e.g. the connection string for the blob storage server) are also set in the tool's config file.

Examples

Saving a file

Saving a file through the blob storage:

await Content.UploadBlobAsync(parentId, fileName, fileLength, async (contentId, versionId, uploadToken) =>
{
    // load a stream from the file system
    using (var localStream = File.OpenRead(filePath))
    {
        // write it directly to the blob storage
        await BlobStorageClient.CopyFromStreamAsync(versionId, uploadToken, localStream);
    }
});

You have to provide the necessary environment values (parent folder id, content name, length) and an action to perform when writing to the blob storage directly. You may use this place to log or take other steps, but generally you can use the code above as is, without changes. The client api hides the start and finalize requests from you, so you will not have to bother with those requests manually.

Take a look at the other methods of the BlobStorageClient class (that is the entry point for client blob operations) for other possibilities like writing chunks or loading readonly streams.

Loading a readonly stream

// get the blob token first from the portal
var token = await Content.GetBlobToken(fileId);
using (var stream = BlobStorageClient.GetStreamForRead(token))
{
    // process the stream...
    await stream.CopyToAsync(target);
}

Related links

References

There are no external references for this article.