Adaptive Sensing

An essential trade off within all mobile sensing apps is between (a) data density (high sampling rates) and continuity (no data gaps) and (b) battery drain. It is easy to configure Sensus to optimize either of these objectives in isolation, but striking a balance between them poses significant challenges. For example, one might wish to sample high-density accelerometry when the device is likely to be used in particular ways (e.g., while walking). Similarly, one might wish to sample light and temperature data, but only in particular geographic locations. No single, static sensing configuration would satisfy such objectives while also having minimal impact on the battery. Such use cases require dynamic sensing configurations that adapt to usage contexts.

Adaptive Control

The figure below depicts concepts that drive adaptive sensing within Sensus.

image

The following concepts are indicated:

  • Sensing agent: An entity that controls sensing parameters (e.g., sampling rates and continuity) on the basis of observed state (i.e., <xref:Sensus.Datum> objects).

  • Opportunistic observation: Sensing agents can be configured to observe state information actively or opportunistically. Opportunistic observation captures any <xref:Sensus.Datum> object generated by Sensus during normal operation. In this mode, the app will not take any extra actions to observe data. This has the effect of minimizing battery drain at the expense of weaker state estimates and sensing control.

  • Opportunistic control: In response to opportunistically observed data, the sensing agent may decide to control sensing in a particular way (e.g., by enabling continuous sensing or increasing sampling rates). Such decisions are only possible upon the arrival of opportunistic data.

  • Action interval: In contrast with opportunistic sensing, sensing agents can be configured to actively seek out data. This has the effect of strengthening state estimates and sensing control at the expense of increased battery drain. The action interval indicates how frequently Sensus should actively observe data for state estimation.

  • Active observation duration: Once the action interval elapses, Sensus will begin to actively observe each <xref:Sensus.Datum> generated by the app. This parameter governs how long active observation should continue before checking the control criterion.

  • Control criterion: Regardless of whether Sensus is observing opportunistically or actively, the control criterion defines state estimates that trigger sensing control. For example, a criterion might specify that sensing control should occur when the average acceleration magnitude exceeds a critical threshold. This is demonstrated in the example sensing agent. This agent also demonstrates a control criterion based on proximity of the phone to a surface (e.g., face) and user activities (e.g., walking).

  • Control completion check interval: Once sensing control is invoked, the agent will periodically recheck the control criterion to determine whether it is still met. If the criterion is not met, then sensing control ends, the sensing agent transitions sensing settings as needed (e.g., reducing sampling rates), and the sensing agent returns to its idle state. If the criterion is still met, then sensing control continues unabated until the next completion check occurs. This parameter governs how long the agent should wait between each completion check.

Sensus supports two mechanisms for incorporating sensing agents into a study. The first involves writing software (in C#) to define the sensing agent. The second involves defining the agent in a JSON-based specification language. The following sections provide more information about these two alternatives.

Software-Defined Sensing Agents

Android

On Android, Sensus supports a plug-in architecture for software-defined sensing agents. This architecture is intended to support research into adaptive sensing by providing a simple interface through which researchers can deploy agents that implement specific adaptation approaches. Follow the steps below to implement and deploy an Android sensing agent.

  1. Create a new Android Class Library project in Visual Studio. In Visual Studio for Mac, the following image shows the correct selection:

image

  1. Add a NuGet reference to the Sensus package.

  2. Add a new class that inherits from <xref:Sensus.Adaptation.SensingAgent>. Be sure to provide a parameterless constructor for your class, as this constructor will be called at run time to create your agent. Your class will be required to override a few methods related to sensing control. These methods are where your sensing agent should execute its control policy. The <xref:Sensus.Adaptation.SensingAgent> class provides a set of predefined control criterion functions here. You can call these diretly from your code or write your own functions to suit your adaptation requirements.

  3. Build the library project, and upload the resulting .dll to a web-accessible URL. A convenient solution is to upload the .dll to a Dropbox directory and copy the sharing URL for the .dll file.

  4. Generate a QR code that points to your .dll (e.g., using QR Code Generator). The content of the QR code must be exactly as shown below:

sensing-agent:URL

where URL is the web-accessible URL that points to your .dll file. If you are using Dropbox, then the QR code content will be similar to the one shown below (note the dl=1 ending of the URL, and note that the following URL is only an example -- it is not actually valid):

sensing-agent:https://www.dropbox.com/s/dlaksdjfasfasdf/SensingAgent.dll?dl=1

  1. In your <xref:Sensus.Protocol> settings, tap "Set Agent" and scan your QR code. Sensus will fetch your .dll file and extract any agent definitions contained therein. Select your desired agent.

  2. Continue with configuration and distribution of your protocol. When run, your protocol will invoke the selected sensing agent according to the state diagram shown above.

See the following implementation for an example:

  • Acceleration (code here): A sensing agent that samples continuously if the device is moving or near a surface (e.g., face).

iOS

In contrast with Android, iOS does not allow apps to load code (e.g., from the above .dll assembly) at run time. Thus, software-defined adaptive sensing agents are more limited on iOS compared with Android. Here are the options:

  • The app comes with one example sensing agent (code here); however, this is simply for demonstration and is unlikely to work well in practice. Nonetheless, you can select this agent when configuring the <xref:Sensus.Protocol>.

  • You can redeploy Sensus as your own app, to which you can add your own agent implementations.

  • You can implement your own agent implementations following the instructions above for Android and email our team (uva.ptl@gmail.com) to include your implementation in a future release of the iOS app.

Adaptive Sensing Policy Language (ASPL) Defined Sensing Agents

In addition to the software-defined adaptive sensing agents described above, Sensus supports the definition of sensing agents in a general-purpose adaptive sensing policy language (ASPL). ASPL specifies both the control criteria as well as the control actions depicted in the above state diagram. The example ASPL policy file demonstrates the ASPL format. The elements of the format are described in the documentation for <xref:Sensus.Adaptation.SensingAgent> and <xref:Sensus.Adaptation.AsplSensingAgent>. If more than 1 <xref:Sensus.Adaptation.AsplStatement> is provided to the <xref:Sensus.Adaptation.AsplSensingAgent>, then the first one whose criterion is satisfied by the observed data will be used for sensing control.

In the example ASPL policy file, you will see many places where property types and property names are specified. In general, each <xref:Sensus.Adaptation.AsplElement> will specify a property type that is the fully-qualified name of a <xref:Sensus.Datum> sub-type. The fully-qualified type name is the concatenation of the type's namespace and name. For example, the <xref:Sensus.Probes.Movement.AccelerometerDatum> type has namespace Sensus.Probes.Movement and name AccelerometerDatum. The fully-qualified type name is thus Sensus.Probes.Movement.AccelerometerDatum. You can find the namespace for each type in the API documentation (e.g., see here). The API documentation page for each type also lists the properties available for that type (e.g., here). So, for example, if you wish to specify the <xref:Sensus.Probes.Movement.AccelerometerDatum.X> property within the <xref:Sensus.Adaptation.AsplElement>, then the property type should be Sensus.Probes.Movement.AccelerometerDatum and the property name should be X.

The other places where property types and property names appear in ASPL are the <xref:Sensus.Adaptation.AsplStatement.BeginControlSettings> and <xref:Sensus.Adaptation.AsplStatement.EndControlSettings>. These types and names refer to the <xref:Sensus.Probes.Probe> properties that should be changed to begin and end sensing control. For example, you may wish to begin control by enabling the <xref:Sensus.Probes.Movement.AccelerometerProbe> with a sampling rate of 50 Hz. This involves two settings: (1) enabling the probe, and (2) setting the sampling rate. Each of these is specified within a <xref:Sensus.ProtocolSetting> in the ASPL JSON.

Softare- Versus ASPL-Defined Sensing Agents

There are pros and cons of software- and ASPL-defined sensing agents:

  • Software-Defined

    • Pros: Sophistication of control criteria. Actions are not limited to the logical structure of ASPL. Any criterion that can be implemented in C# would be feasible.
    • Cons: Low-level programming is required. Third-party deployment of iOS agents is complicated by iOS's prohibition of run-time code loading (see above). Changing the agent definition (whether on Android or iOS) involves modifying code and, for iOS, redeploying the application.

  • ASPL-Defined

    • Pros: Agent definitions use the relatively simple ASPL syntax. Agent definitions can be loaded at run-time into both Android and iOS without the need for code changes or app redeployment.
    • Cons: ASPL has limited logical expressiveness.

Distributing Sensing Agent Policies

Regardless of whether a software- or ASPL-defined sensing agent is used, a policy must be provided to the agent, specifying the agent's control parameters. This can be done in two ways:

  • Set within protocol: In the protocol settings, tap "Set Agent Policy", then select your JSON policy file. This works well to set the initial policy used by the sensing agent; however, this is not a very effective means of providing updated sensing policies to users during an ongoing study. Updating the policy would require the study administrator to edit the protocol and distribute the new protocol to all users (e.g., via email), who would then need to manually update their protocols.

  • Send via push notification: Request a push notification update with the type set to <xref:Sensus.Notifications.PushNotificationUpdateType.SensingAgentPolicy> and content set to the policy you wish to provide. Sensus will parse the content into a JSON object and pass the resulting object to your agent via <xref:Sensus.Adaptation.SensingAgent.ProtectedSetPolicyAsync>. This is an effective option for updating the sensing agent's policy during ongoing studies, as users will not need to do anything in order to receive the updated policies.

Testing and Debugging

Regardless of whether your sensing agent targets Android or iOS, there are a few ways to test and debug it:

  • Monitor the agent state: Within your <xref:Sensus.Protocol> settings, tap "View Agent State" to see a real-time display of your agent's state. You will see it cycle through the state diagram shown above.

  • Write to the log file: See the code for the example agents above. You will see calls that write to the log file. Use similar calls in your code to write information about the behavior of your agent to the log. Run your agent for a while in the app and then share the log file with yourself from within the app. Note that the size of the log file is limited, so you might not be able to view the entire log history of your agent.

  • Flash notifications on the screen: On Android, you can flash notifications on the screen as shown in the example code. These messages will appear for a short duration.

  • Run your agent in the debugger: By far the most useful approach is to configure a development system and run Sensus in the debugger with your sensing agent. You will need to add your agent code to the Sensus app projects in order to step through it in the debugger.

Known Limitations and Future Improvements

  • Initiation of continuous sensing from the background on iOS: iOS places significant constraints on Sensus's ability to operate in the background. This impacts all sensing agents' ability to initiate continuous sensing from the background. A sensing agent will be able to update its state estimtes from the background upon receipt of a push notification; however, there is no known way to initiate continuous background operation while in the background state. As a result, if the agent's state estimate indicates that continuous sensing control is warranted, then this control will not be initiated until the next time the app is brought to the foreground by the user. It would be useful to add the option of notifying the user from the background when continuous sensing is requested by the sensing agent.

  • Termination of sensing control: Currently, sensing control (whether opportunistic or active) can only be terminated after the control completion check interval elapses. This is probably too coarse, and it would be helpful to complement this check with checks run after a certain number of data readings have been observed while in a control state.

Data Streams

In addition to directly impacting the data streams that are collected (e.g., via sampling rates and enabling/disabling [probes](<xref:Sensus.Probes.Probe)), the use of a sensing agent will cause additional data to be written to the local data store. These include:

  • Datum-level tagging: Each <xref:Sensus.Datum> collected by the app will be tagged with a description of the sensing agent's state at the time when the <xref:Sensus.Datum> was collected. This is achieved by setting <xref:Sensus.Datum.SensingAgentStateDescription> to the value of <xref:Sensus.Adaptation.SensingAgent.StateDescription>.

  • Sensing agent lifecycle: Each time the sensing agent transitions from one state to another, a <xref:Sensus.Adaptation.SensingAgentStateDatum> will be written to the local data store to record the transition. This will be done regardless of whether any other <xref:Sensus.Datum> readings are collected.