This page last changed on May 05, 2011 by ebariaux.

In the current implementation of a button, only one action can be associated with it. It is executed immediately when the user presses the button.
Optionally, it can be marked as repeat, so that the action is executed every x ms (300 currently for iOS) as long as the user keeps pressing the button.

We would need to extend that to support additional forms of interaction with the button:

  • release : it should be possible to trigger a specific action when the user releases the button. Potentially, if no action is defined on press, this means the action is only triggered when the user releases the button. Defining both can be used for instance to dim a light for system supporting a start dim and stop dim command.
  • long press : if the user keeps pressing the button for some time, another different action is executed (and the short press action is not). Defining a long press action would prevent use of the repeat option.
  • long release : same as short release
  • double tap : some systems support that, not sure we need it as in general defining too many control options on one button is not user friendly.

To sum it up, it will be possible to define 4 actions on a button:

  • on short press, with option to repeat if no long press/release action defined and configurable delay for repeat
  • on short release
  • on long press, with configurable delay for what "long" means
  • on long release

In this approach, the logic to map between a specific interaction and a command to send is implemented in the console. Nothing needs to be changed as far as communication protocol (REST command) with the controller.

Another approach could be to send button events to the controller (button pushed, button release) and let the controller decide on how this maps to a command. This mandates usage of some form of scripting on the controller side which 1. is not fully implemented yet and 2. would make configuration more difficult (though it is always possible to have the modeler generate the script from a more user friendly configuration UI). Anyway, I don't believe this second option is practical at this stage.

In this approach, the logic to map between a specific interaction and a command to send is implemented in the console. Nothing needs to be changed as far as communication protocol (REST command) with the controller.

This might not be that easy, the mechanism is as follows:
If a command is defined on a button, the <button> tag in panel.xml has the hasControlCommand attribute set to true.
When the button is pressed, the console calls rest/control/<button id>/click to the controller.
The controller will look up the received id (id of the button) and create a list of commands to execute.

So the protocol might stay the same, replacing the last part (click) with specific commands (press, release, long_press, long_release, repeat) but both the panel and controller XML need to be updated and the way the controller processes this needs to be reviewed.

Posted by ebariaux at May 05, 2011 16:41

Let's tackle first the panel.xml changes.
Current schema for button is

	<xsd:element name="button">
		<xsd:annotation>
			<xsd:documentation><![CDATA[
				The button on panel.Its label is the name shown on the button.
				Its position is defined by coordinates (x, y).
				Its size is defined by width and height.The meature is one cell on the panel.
				attribute repeat: whether the button send command on and on when it is pressed down. 	
				attribute hasControlCommand: whether the button has bound a command. 			
			]]></xsd:documentation>
		</xsd:annotation>
		<xsd:complexType>
			<xsd:sequence>
				<xsd:group ref="buttonGroup" minOccurs="0" maxOccurs="unbounded"></xsd:group>
			</xsd:sequence>
			<xsd:attribute name="id" type="idType" use="required" />
			<xsd:attribute name="name" type="xsd:string" use="required" />
			<xsd:attribute name="repeat" type="xsd:boolean" use="optional"/>
			<xsd:attribute name="hasControlCommand" type="xsd:boolean" use="optional"/>
		</xsd:complexType>
	</xsd:element>

This can be enhanced to

	<xsd:element name="button">
		<xsd:annotation>
			<xsd:documentation><![CDATA[
				The button on panel.Its label is the name shown on the button.
				Its position is defined by coordinates (x, y).
				Its size is defined by width and height.The meature is one cell on the panel.
				attribute repeat: whether the button send command on and on when it is pressed down. 	
				attribute hasControlCommand: whether the button has bound a command. 			
			]]></xsd:documentation>
		</xsd:annotation>
		<xsd:complexType>
			<xsd:sequence>
				<xsd:group ref="buttonGroup" minOccurs="0" maxOccurs="unbounded"></xsd:group>
			</xsd:sequence>
			<xsd:attribute name="id" type="idType" use="required" />
			<xsd:attribute name="name" type="xsd:string" use="required" />
			<xsd:attribute name="repeat" type="xsd:boolean" use="optional"/>
			<xsd:attribute name="repeatDelay" type="xsd:integer" minInclusive="100" use="optional"/>
			<xsd:attribute name="hasShortPressCommand" type="xsd:boolean" use="optional"/>
			<xsd:attribute name="hasShortReleaseCommand" type="xsd:boolean" use="optional"/>
			<xsd:attribute name="hasLongPressCommand" type="xsd:boolean" use="optional"/>
			<xsd:attribute name="hasLongReleaseCommand" type="xsd:boolean" use="optional"/>
			<xsd:attribute name="longPressDelay" type="xsd:integer" minInclusive="250" use="optional"/>
		</xsd:complexType>
	</xsd:element>

The repeat stays as is but is considered false whatever the provided value if either hasLongPressCommand or hasLongReleaseCommand is true.
The repeatDelay attribute is added, indicating the delay in milliseconds between successive send of commands. It is ignored if repeat is false.
The hasControlCommand has been replaced with hasShortPressCommand, hasShortReleaseCommand, hasLongPressCommand and hasLongReleaseCommand. Each indicates the presence of a command for the specified event type. See below for impact on REST API.
The longPressDelay is added, indicating the delay in milliseconds before a press is considered long.

Posted by ebariaux at May 06, 2011 11:02

Next, impact on the REST API:

In the current implementation, when the button is pressed, the console calls the following URL: <server>rest/control/<button id>/click on the controller.
If it is a repeat button, it does so every x ms.

Given the proposed changes in panel.xml above, it the API can easily be changed so that the last part of the URL click is replaced with shortPress, shortRelease, longPress and longRelease.

Posted by ebariaux at May 06, 2011 11:10

Controller implementation and controller.xml:

When the controller receives a rest/control URL, it parses the component id and the command (the click part).
Based on those, it then looks up a Control instance and ask it its list of executableCommands.
The Control instance will also validate the command that it understand.
In this case, the Control is a Button and the only allowed command is click.

In the controller.xml, this translates into something like

    <button id="245">
      <include type="command" ref="290" />
      <delay>250</delay>
      <include type="command" ref="289" />
    </button>

In the current code, the list of executableCommands returned by a control is not dependent on the command.
I would propose to enhance this to add this possibility.

So at the code level, instead of using control.getExecutableCommands() the code would be control.getExecutableCommands(command).
In the XML, the definition could be something like

    <button id="245">
      <commands command="shortPress">
        <include type="command" ref="290" />
        <delay>250</delay>
        <include type="command" ref="289" />
      </commands>
      <commands command="longPress">
        <include type="command" ref="291" />
      </commands>
    </button>
Posted by ebariaux at May 06, 2011 11:25

release : it should be possible to trigger a specific action when the user releases the button. Potentially, if no action is defined on press, this means the action is only triggered when the user releases the button. Defining both can be used for instance to dim a light for system supporting a start dim and stop dim command.

Agreed.

long press : if the user keeps pressing the button for some time, another different action is executed (and the short press action is not). Defining a long press action would prevent use of the repeat option.

Agreed.

long release : same as short release

Agreed.

double tap : some systems support that, not sure we need it as in general defining too many control options on one button is not user friendly.

I'm split on this – you're right that too many control options is confusing, and double-tap isn't necessarily needed on touch-panel paradigm. But my initial reaction was to include double-tap too.

Posted by juha at May 10, 2011 13:26

So the protocol might stay the same, replacing the last part (click) with specific commands (press, release, long_press, long_release, repeat)

Agree.

Posted by juha at May 10, 2011 14:44

Re-reading that, there is an issue in what I wrote. If you define a long press action, there is an issue with the way the short press works.
The only way for the system to be sure the press is short is to detect that the release has happened before the long press delay. In that case, the short press event and the short release event will in effect happen at the same time.
If no long press is defined on the button, then the short press event can happen immediately when the user taps on the button.

Posted by ebariaux at May 10, 2011 15:38

Yeah I was stuck with the same thing, which is why I kept editing and re-editing my response.

That is, if you want the same button to react differently to two user inputs, you need to define short-release in combination with long-press (or even long-release).

But I think short-press in combination with long-press (with no releases) will yield a confusing behavior.

Posted by juha at May 10, 2011 16:22

But I think short-press in combination with long-press (with no releases) will yield a confusing behavior.

Although maybe there's a use case for this too : accelerating press

Posted by juha at May 10, 2011 16:26

Agreed on the schema proposal.

Posted by juha at May 10, 2011 16:34

Yes.

Posted by juha at May 10, 2011 16:47

Makes sense. The control structure in the controller itself may still need to go through refactoring but the proposal makes sense in terms of how things are currently implemented.

Posted by juha at May 10, 2011 16:53

My current conclusion is that as soon as the user touches the button, a press action is executed (so the name is press, not shortPress because we cannot know if it will be short or long).
This press action is executed irrelevant of the fact that the press will be short or long.

Posted by ebariaux at May 19, 2011 09:41

How does it handle the typical long press that exists on Android in this case (or would it not) ?

That is, I press down, and after holding down for x seconds, an action is triggered (i.e before there's release).

Something still nags me about using press and release as the only primitives, rather than something that encompasses both (like 'tap' or 'long press')

Posted by juha at May 19, 2011 10:00

Something still nags me about using press and release as the only primitives, rather than something that encompasses both (like 'tap' or 'long press')

Or maybe they work as primitives, but at tooling level I think we may need to expose them as something like 'tap' or 'long press' – but we may not need to know what those are yet, and simply name them as we use them and find the combinations that make sense.

Is the delay a property of release since press is immediate?

Posted by juha at May 19, 2011 10:05

There is still a long press action. Looking at the attributes defined on the button (adapted from above)

<xsd:attribute name="repeat" type="xsd:boolean" use="optional"/>
<xsd:attribute name="repeatDelay" type="xsd:integer" minInclusive="100" use="optional"/>
<xsd:attribute name="hasPressCommand" type="xsd:boolean" use="optional"/>
<xsd:attribute name="hasShortReleaseCommand" type="xsd:boolean" use="optional"/>
<xsd:attribute name="hasLongPressCommand" type="xsd:boolean" use="optional"/>
<xsd:attribute name="hasLongReleaseCommand" type="xsd:boolean" use="optional"/>
<xsd:attribute name="longPressDelay" type="xsd:integer" minInclusive="250" use="optional"/>

we can have the following use cases:
1. Have a button with

hasPressCommand="true" repeat="true" repeatDelay="200" hasShortReleaseCommand="true"
. The user touches the button, press command is send immediately, then repeatedly every 200ms. Short release command is send when user stops touching button.
2. Have a button with
hasPressCommand="true" hasShortReleaseCommand="true" hasLongPressCommand="true" hasLongReleaseCommand="true" longPressDelay="500"
. The following 2 scenarios are possible.
a. User touches button, press command is send. User stops touching button after 400ms, short release command is sent.
b. User touches button, press command is send. After 500ms, user is still touching button, longPressCommand is sent. After more than 500ms, user stops touching button, long release command is sent.

Posted by ebariaux at May 19, 2011 19:28

Let's go with that.

In terms of tooling, I would still think about presenting it as a configuration of 'tap + long press' which would translate to:

hasShortReleaseCommand="true" hasLongPressCommand="true" longPressDelay="500"

if I understood correctly.

Posted by juha at May 19, 2011 20:03

To sum it up, what is implemented is:

Changes to the panel schema
	<xsd:element name="button">
		<xsd:annotation>
			<xsd:documentation><![CDATA[
				The button on panel.Its label is the name shown on the button.
				Its position is defined by coordinates (x, y).
				Its size is defined by width and height.The meature is one cell on the panel.
				attribute repeat: whether the button send command on and on when it is pressed down. 	
				attribute hasControlCommand: whether the button has bound a command. 			
			]]></xsd:documentation>
		</xsd:annotation>
		<xsd:complexType>
			<xsd:sequence>
				<xsd:group ref="buttonGroup" minOccurs="0" maxOccurs="unbounded"></xsd:group>
			</xsd:sequence>
			<xsd:attribute name="id" type="idType" use="required" />
			<xsd:attribute name="name" type="xsd:string" use="required" />
			<xsd:attribute name="repeat" type="xsd:boolean" use="optional"/>
			<xsd:attribute name="repeatDelay" type="xsd:integer" minInclusive="100" use="optional"/>
			<xsd:attribute name="hasShortPressCommand" type="xsd:boolean" use="optional"/>
			<xsd:attribute name="hasShortReleaseCommand" type="xsd:boolean" use="optional"/>
			<xsd:attribute name="hasLongPressCommand" type="xsd:boolean" use="optional"/>
			<xsd:attribute name="hasLongReleaseCommand" type="xsd:boolean" use="optional"/>
			<xsd:attribute name="longPressDelay" type="xsd:integer" minInclusive="250" use="optional"/>
		</xsd:complexType>
	</xsd:element>
Changes to the controller schema

The button now embeds 1 to 4 macros (instead of directly embedding commands), each one qualified with the appropriate commandParam

  <xsd:element name = "button">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element ref = "macro" minOccurs = "0" maxOccurs = "4"/>
      </xsd:sequence>
      <xsd:attribute name = "id" type = "idType" use = "required" />
    </xsd:complexType>
  </xsd:element>
and
  <xsd:element name = "macro">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:group ref = "macroElement" minOccurs="0" maxOccurs="unbounded"/>
      </xsd:sequence>
      <xsd:attribute name = "commandParam" type = "xsd:string" use = "required" />
    </xsd:complexType>
  </xsd:element>

Changes to the REST API

The last part of the call is now either press, shortRelease, longPress or longRelease.

Posted by ebariaux at May 24, 2011 12:56

Hi.

Has any of this been implemented. It is very old and I have not seen anything (like a short/long press) in any example.

Posted by niraviry at Nov 18, 2014 17:17
Document generated by Confluence on Jun 05, 2016 09:30