Class AmazonS3RemoteDataStore

The Amazon S3 Remote Data Store allows Sensus to upload data from the device to Amazon's Simple Storage Service (S3). The S3 service is a simple, non-relational storage system that is relatively cheap, easy to use, and robust.

Prerequisites

Sign up for an account with Amazon Web Services, if you don't have one already. The Free Tier is sufficient.
Install the AWS Command Line Interface(CLI).
Download and unzip our AWS configuration scripts.
Run the following command to configure an S3 bucket for use within a Sensus Amazon S3 Remote Data Store, where NAME is an informative name (alphanumerics and dashes only) and REGION is the region in which your bucket will reside (e.g., us-east-1):

./configure-s3.sh NAME REGION

The previous command will create a bucket as well as an IAM group and user with write-only access to the bucket. If successful, the command will output something like the following:

Done. Details:
Sensus S3 bucket:  test-bucket-eee8ef46-5d6a-4508-b745-e6635d195a85
Sensus S3 IAM account:  XXXX:XXXX

The bucket and IAM account produced on the final line should be kept confidential. Use these values as Bucket and IamAccountString, respectively.

Downloading Data from Amazon S3

Install the AWS Command Line Interface. Assuming you have created and populated an S3 bucket named BUCKET and a folder named FOLDER, you can download all of your Sensus data in a few different ways:

You can use the functions (e.g., sensus.sync.from.aws.s3) in the SensusR package.
You can execute the following command to download everything to a directory named data on your desktop:
```
aws s3 cp --recursive s3://BUCKET/FOLDER ~/data
```
You can use a third-party application like Bucket Explorer to browse and download data from Amazon S3.

Deconfiguration

If you are finished collecting data and you would like to prevent any future data submission, you can deconfigure the IAM group and user with the following command, where BUCKET corresponds to the Sensus S3 bucket name created above:

./deconfigure-s3.sh BUCKET

The preceding command will not delete your bucket or data.

Inheritance

System.Object

DataStore

RemoteDataStore

AmazonS3RemoteDataStore

Inherited Members

RemoteDataStore.StopAsync()

RemoteDataStore.WriteLocalDataStoreAsync(CancellationToken)

RemoteDataStore.WriteDelayMS

RemoteDataStore.WriteTimeoutMinutes

RemoteDataStore.WriteOnPowerConnect

RemoteDataStore.RequireWiFi

RemoteDataStore.RequireCharging

RemoteDataStore.RequiredBatteryChargeLevelPercent

RemoteDataStore.UserNotificationMessage

RemoteDataStore.DelayToleranceBeforeMS

RemoteDataStore.DelayToleranceAfterMS

DataStore.RestartAsync()

DataStore.Copy()

DataStore.Running

DataStore.Protocol

DataStore.CaptionText

DataStore.UpdatedCaptionText

Namespace: Sensus.DataStores.Remote

Assembly: SensusAndroid.dll

Syntax

public class AmazonS3RemoteDataStore : RemoteDataStore

Constructors

AmazonS3RemoteDataStore()

Declaration

public AmazonS3RemoteDataStore()

Fields

ADAPTIVE_EMA_POLICIES_DIRECTORY

The adaptive EMA policies directory.

Declaration

public const string ADAPTIVE_EMA_POLICIES_DIRECTORY = "adaptive-ema-policies"

Field Value

Type	Description
System.String

DATA_DIRECTORY

The data directory.

Declaration

public const string DATA_DIRECTORY = "data"

Field Value

Type	Description
System.String

PUSH_NOTIFICATIONS_DIRECTORY

The push notifications directory.

Declaration

public const string PUSH_NOTIFICATIONS_DIRECTORY = "push-notifications"

Field Value

Type	Description
System.String

PUSH_NOTIFICATIONS_REQUESTS_DIRECTORY

The push notifications requests directory.

Declaration

public const string PUSH_NOTIFICATIONS_REQUESTS_DIRECTORY = "push-notifications/requests"

Field Value

Type	Description
System.String

PUSH_NOTIFICATIONS_TOKENS_DIRECTORY

The push notifications tokens directory.

Declaration

public const string PUSH_NOTIFICATIONS_TOKENS_DIRECTORY = "push-notifications/tokens"

Field Value

Type	Description
System.String

PUSH_NOTIFICATIONS_UPDATES_DIRECTORY

The push notifications updates directory.

Declaration

public const string PUSH_NOTIFICATIONS_UPDATES_DIRECTORY = "push-notifications/updates"

Field Value

Type	Description
System.String

SENSING_POLICIES_DIRECTORY

The sensing policies directory.

Declaration

public const string SENSING_POLICIES_DIRECTORY = "sensing-policies"

Field Value

Type	Description
System.String

Properties

Bucket

The AWS S3 bucket in which data should be stored. This is the bucket identifier output by the steps described in the summary for this class.

Declaration

[EntryStringUiProperty(null, true, 2, true)]
public string Bucket { get; set; }

Property Value

Type	Description
System.String	The bucket.

CanRetrieveWrittenData

Declaration

public override bool CanRetrieveWrittenData { get; }

Property Value

Type	Description
System.Boolean

Overrides

RemoteDataStore.CanRetrieveWrittenData

DisplayName

Declaration

public override string DisplayName { get; }

Property Value

Type	Description
System.String

Overrides

DataStore.DisplayName

Folder

The folder within Bucket where data should be stored.

Declaration

[EntryStringUiProperty(null, true, 3, false)]
public string Folder { get; set; }

Property Value

Type	Description
System.String	The folder.

IamAccountString

The IAM user's access and secret keys output by the steps described in the summary for this class.

Declaration

[EntryStringUiProperty("IAM Account:", true, 4, true)]
public string IamAccountString { get; set; }

Property Value

Type	Description
System.String	The iam account string.

PinnedPublicKey

Pinned SSL public encryption key associated with PinnedServiceURL. Use this to set up SSL certificate pinning.

Declaration

[EntryStringUiProperty("Pinned Public Key:", true, 6, false)]
public string PinnedPublicKey { get; set; }

Property Value

Type	Description
System.String	The pinned public key.

PinnedServiceURL

Alternative URL to use for S3, instead of the default. Use this to set up SSL certificate pinning.

Declaration

[EntryStringUiProperty("Pinned Service URL:", true, 5, false)]
public string PinnedServiceURL { get; set; }

Property Value

Type	Description
System.String	The pinned service URL.

Region

The AWS region in which Bucket resides (e.g., us-east-2).

Declaration

[ListUiProperty(null, true, 1, new object[]{"us-east-1", "us-east-2", "us-west-1", "us-west-2", "ca-central-1", "ap-south-1", "ap-northeast-2", "ap-southeast-1", "ap-southeast-2", "ap-northeast-1", "eu-central-1", "eu-west-1", "eu-west-2", "sa-east-1", "us-gov-west-1"}, true)]
public string Region { get; set; }

Property Value

Type	Description
System.String	The region.

StorageDescription

Declaration

public override string StorageDescription { get; }

Property Value

Type	Description
System.String

Overrides

RemoteDataStore.StorageDescription

Methods

DeletePushNotificationRequestAsync(Guid, CancellationToken)

Declaration

public override Task DeletePushNotificationRequestAsync(Guid backendKey, CancellationToken cancellationToken)

Parameters

Type	Name	Description
System.Guid	backendKey
System.Threading.CancellationToken	cancellationToken

Returns

Type	Description
System.Threading.Tasks.Task

Overrides

RemoteDataStore.DeletePushNotificationRequestAsync(Guid, CancellationToken)

DeletePushNotificationTokenAsync(CancellationToken)

Declaration

public override Task DeletePushNotificationTokenAsync(CancellationToken cancellationToken)

Parameters

Type	Name	Description
System.Threading.CancellationToken	cancellationToken

Returns

Type	Description
System.Threading.Tasks.Task

Overrides

RemoteDataStore.DeletePushNotificationTokenAsync(CancellationToken)

GetDatumAsync<T>(String, CancellationToken)

Declaration

public override Task<T> GetDatumAsync<T>(string datumKey, CancellationToken cancellationToken)
    where T : Datum

Parameters

Type	Name	Description
System.String	datumKey
System.Threading.CancellationToken	cancellationToken

Returns

Type	Description
System.Threading.Tasks.Task<T>

Type Parameters

Name	Description
T

Overrides

RemoteDataStore.GetDatumAsync<T>(String, CancellationToken)

GetDatumKey(Datum)

Declaration

public override string GetDatumKey(Datum datum)

Parameters

Type	Name	Description
Datum	datum

Returns

Type	Description
System.String

Overrides

RemoteDataStore.GetDatumKey(Datum)

GetPushNotificationUpdatesAsync(CancellationToken)

Declaration

public override Task<List<PushNotificationUpdate>> GetPushNotificationUpdatesAsync(CancellationToken cancellationToken)

Parameters

Type	Name	Description
System.Threading.CancellationToken	cancellationToken

Returns

Type	Description
System.Threading.Tasks.Task<System.Collections.Generic.List<PushNotificationUpdate>>

Overrides

RemoteDataStore.GetPushNotificationUpdatesAsync(CancellationToken)

GetScriptAgentPolicyAsync(CancellationToken)

Gets the policy for the IScriptProbeAgent from the current AmazonS3RemoteDataStore. This method will download the policy JSON object from the following location:

BUCKET/DIRECTORY/DEVICE

where BUCKET is Bucket, DIRECTORY is the value of ADAPTIVE_EMA_POLICIES_DIRECTORY, and DEVICE is the identifier of the current device as returned by DeviceId. This is the same device identifier used within all Datum objects generated by the current device. This is also the same device identifier stored in all JSON objects written to the AmazonS3RemoteDataStore. To provide a policy JSON object, write the policy JSON object to the above S3 location. The content of this S3 location will be read, parsed as Newtonsoft.Json.Linq.JObject, and delivered to the IScriptProbeAgent via SetPolicyAsync(JObject).

Declaration

public override Task<JObject> GetScriptAgentPolicyAsync(CancellationToken cancellationToken)

Parameters

Type	Name	Description
System.Threading.CancellationToken	cancellationToken	Cancellation token.

Returns

Type	Description
System.Threading.Tasks.Task<Newtonsoft.Json.Linq.JObject>	The script agent policy.

Overrides

RemoteDataStore.GetScriptAgentPolicyAsync(CancellationToken)

GetSensingAgentPolicyAsync(CancellationToken)

Gets the policy for the Agent from the current AmazonS3RemoteDataStore. This method will download the policy JSON object from the following location:

BUCKET/DIRECTORY/DEVICE

where BUCKET is Bucket, DIRECTORY is the value of SENSING_POLICIES_DIRECTORY, and DEVICE is the identifier of the current device as returned by DeviceId. This is the same device identifier used within all Datum objects generated by the current device. This is also the same device identifier stored in all JSON objects written to the AmazonS3RemoteDataStore. To provide a policy JSON object, write the policy JSON object to the above S3 location. The content of this S3 location will be read, parsed as Newtonsoft.Json.Linq.JObject, and delivered to the Agent via SetPolicyAsync(JObject).

Declaration

public override Task<JObject> GetSensingAgentPolicyAsync(CancellationToken cancellationToken)

Parameters

Type	Name	Description
System.Threading.CancellationToken	cancellationToken	Cancellation token.

Returns

Type	Description
System.Threading.Tasks.Task<Newtonsoft.Json.Linq.JObject>	The sensing agent policy.

Overrides

RemoteDataStore.GetSensingAgentPolicyAsync(CancellationToken)

Reset()

Declaration

public override void Reset()

Overrides

RemoteDataStore.Reset()

SendPushNotificationRequestAsync(PushNotificationRequest, CancellationToken)

Declaration

public override Task SendPushNotificationRequestAsync(PushNotificationRequest request, CancellationToken cancellationToken)

Parameters

Type	Name	Description
PushNotificationRequest	request
System.Threading.CancellationToken	cancellationToken

Returns

Type	Description
System.Threading.Tasks.Task

Overrides

RemoteDataStore.SendPushNotificationRequestAsync(PushNotificationRequest, CancellationToken)

SendPushNotificationTokenAsync(String, CancellationToken)

Declaration

public override Task SendPushNotificationTokenAsync(string token, CancellationToken cancellationToken)

Parameters

Type	Name	Description
System.String	token
System.Threading.CancellationToken	cancellationToken

Returns

Type	Description
System.Threading.Tasks.Task

Overrides

RemoteDataStore.SendPushNotificationTokenAsync(String, CancellationToken)

StartAsync()

Declaration

public override Task StartAsync()

Returns

Type	Description
System.Threading.Tasks.Task

Overrides

RemoteDataStore.StartAsync()

TestHealthAsync(List<AnalyticsTrackedEvent>)

Declaration

public override Task<HealthTestResult> TestHealthAsync(List<AnalyticsTrackedEvent> events)

Parameters

Type	Name	Description
System.Collections.Generic.List<AnalyticsTrackedEvent>	events

Returns

Type	Description
System.Threading.Tasks.Task<HealthTestResult>

Overrides

RemoteDataStore.TestHealthAsync(List<AnalyticsTrackedEvent>)

WriteDataStreamAsync(Stream, String, String, CancellationToken)

Declaration

public override Task WriteDataStreamAsync(Stream stream, string name, string contentType, CancellationToken cancellationToken)

Parameters

Type	Name	Description
System.IO.Stream	stream
System.String	name
System.String	contentType
System.Threading.CancellationToken	cancellationToken

Returns

Type	Description
System.Threading.Tasks.Task

Overrides

RemoteDataStore.WriteDataStreamAsync(Stream, String, String, CancellationToken)

WriteDatumAsync(Datum, CancellationToken)

Declaration

public override Task WriteDatumAsync(Datum datum, CancellationToken cancellationToken)

Parameters

Type	Name	Description
Datum	datum
System.Threading.CancellationToken	cancellationToken

Returns

Type	Description
System.Threading.Tasks.Task

Overrides

RemoteDataStore.WriteDatumAsync(Datum, CancellationToken)