This page last changed on Nov 13, 2011 by kurrazyman.

I have been working on a connection manager implementation for quite some time and I have finally got round to making it compatible with the existing controller.xml without any modifications needed to the controller.xml (although to get the most out of this branch then you should look at modifying the controller.xml).

What does it do?

This code, uses a connection manager for each connection to a server.

In the existing code each time a command needs to be sent to the server or a sensor value needs to be updated a new connection is made to the server and after the command has been executed the server is then disconnected. If you have lots of sensors that connect to the same server you quickly run into issues as multiple simultaneous connections are made to the same server and commands fail to execute.

The connection manager introduced here, does exactly what it says on the tin. It manages the connection to the individual servers and then all commands are routed through this connection. At present a permanent connection is made to the server so when the controller starts it opens the connection and when the controller is stopped it closes the connection.

The connection manager also supports command scripting (currently JavaScript) and multiple command actions (a single command can consist of any number of command send, read or script Actions; send actions send a message to the server, read actions read data from the server and script actions run a script that allows you to process the data retrieved from the server and convert it into the correct format for updating a sensor value for example.

Making use of the scripting functionality requires manual changes to the command definition in the controller.xml; if you are interested in trying this then let me know and I'll give you some pointers, this is all still in development but it does give you a lot of power to write more complicated commands.

What Protocols are supported?

Currently supported protocols: -

  • Telnet
  • UDP
  • HTTP
  • TCP

Any supported protocols in your controller.xml file will be filtered through the connection manager, other command protocols will continue to work as they did before using the old mechanism.

Where can I find it?

http://openremote.svn.sourceforge.net/svnroot/openremote/branches/feature/Controller_2_0_0_Alpha11_Manager/

How do I compile it?

Building code from the repository has been explained in the page below, just checkout the branch listed above: -

http://openremote.org/display/docs/Building+OpenRemote+2.0+Developer+Releases

How do I use it?

Once compiled it should run just like the existing controller but you will notice in the console window some different messages appearing. When one or more commands are found that use a protocol supported by the gateway manager a new gateway connection is created for each unique server (for Telnet it is the host IP address and port that make it unique), when this gateway connection is started a message is output to the console window: -

-------- GATEWAY: Started gateway for (Telnet 192.168.1.71:6601) ThreadThread\-22,5,main

If a connection is lost then the gateway manager will try to re-establish the connection at an increasing interval from 5-60seconds and a message like below will appear in the console window: -

Cannot establish Gateway connection [Telnet 192.168.1.71:6601] so going to sleep for 5 seconds.

Additional Gateway Parameters

The connection manager also accepts some additional gateway parameters that will apply to all commands that use that gateway: -

Parameter Description Value
defaultPollingInterval This sets the default time between polling requests for the sensors that use this gateway. Seconds: 1-86400 (Default = 1)
connectTimeout Sets the time allowed to make the connection to the server. Milliseconds: 1-60000 (Default = 1000)
readTimeout Sets the time allowed to read data from server. Milliseconds: 1-5000 (Default = 200)
sendTerminator Sets the characters used at the end of a send command. String (Default = "\n")

These parameters are optional and if you need them they should be set within the first command element in the controller.xml that the gateway will load, using the standard property Element format, for example: -

    <command id="1" protocol="telnet">
      <property name="port" value="6601" />
      <property name="command" value="OK|status" />
      <property name="ipAddress" value="192.168.1.71" />
      <property name="defaultPollingInterval" value="10000" />
      <property name="connectTimeout" value="20000" />
      <property name="readTimeout" value="1000" />
      <property name="sendTerminator" value="\r" />
    </command>
Additional Command Parameters

The connection manager also accepts some additional command parameters, that need to be set for each command if desired.

Parameter Description Value
pollingInterval This sets the time between polling requests for just this command. Seconds: 1-86400 (Default = 1)

Building on the command element example above if you wanted to also change the polling interval just for that command then: -

    <command id="1" protocol="telnet">
      <property name="port" value="6601" />
      <property name="command" value="OK|status" />
      <property name="ipAddress" value="192.168.1.71" />
      <property name="defaultPollingInterval" value="10000" />
      <property name="connectTimeout" value="20000" />
      <property name="readTimeout" value="1000" />
      <property name="sendTerminator" value="\r" />
      <property name="pollingInterval" value="1000" />
    </command>

Why use this branch?

In it's basic form you will not notice any performance differences with the existing controller, but if you start using scripting and multiple command actions then it opens up another world of functionality. Some reasons you may want to use this code: -

  • If you have been experiencing connection issues with the existing controller because you have lots of sensors that use the same server.
  • You need the connection to be kept open at all times.
  • If you need to format the response from the server before feeding the value back into a sensor.
  • You want to change how often all the sensor values are updated.
  • You want to change how often a single sensor value is updated.

Can this help in my situation?

It you have a particular need that can't be met by the existing code and would like to know if this branch could help, feel free to ask, if it's not currently possible but your situation is not too specific and can benefit others then I may be able to incorporate it for you.


telnet.gif (image/gif)

Hi Richard

Many thanks for supplying this update to Telnet it is working for me now and I have a few questions.

You say in your dOc that there should be additional messages in the consol window, I am not seeing any.

I presume I should also see them in the controler log file as well?

You state in your doc that you can supply some information on scripting to process the data retrieved. I woul

be very interested in this.

At present I am receiving some data back when I send a command but not all that I would expect. Is there a limit on the length of the data string

which is returned?

Many thanks once again for a great job

Pierce

Posted by pjmm at Mar 28, 2011 23:15

Hi Pierce,

Glad you've had a chance to try this code. If everything is working for you, then all you will see in the console window is a line similar to below for each telnet server you are connecting to (The thread numbers will vary): -

-------- GATEWAY: Started gateway for (Telnet 192.168.1.71:6601) Thread Thread\-22,5,main

Assuming you have this then yes the gateway code will be making a single connection to each server.

There is no limit to how much data can be returned (I have being experimenting with the PulseAudio Telnet server which returns about 33K of text for one of my commands), obviously if a large amount of data is being retrieved then this can slow down the response of your panel(s) but there are work arounds that you can use with the gateway code which I can explain if you so wish.

With regards to the command scripting I am currently at work and need to check the code when I get home this evening, once I have done this I will give you some instructions on how to use the scripting functionality. This does mean that you have to manually enter the information into the controller.xml as this can't be done in the online designer at the moment but I'm sure that's not an issue for you esp. having played with my previous modified Telnet branch.

Posted by kurrazyman at Mar 29, 2011 07:23

Hi Richard

I am not seeing any messages when running your script. Neither on screan or in the Log file.

I am als not seeing the messages which I should see from the application I am Telnetting to

Have you any ideas why this should be.

Would it be possable to have a command to disconnect the telnet sesion as this would begood to

be able to end the sesion when finished.

Regards

Pierce

Posted by pjmm at Mar 29, 2011 17:13

Hi Richard

I have connected to my device using Putty and can confirm that it is sending back a string of text

as expected so what I am connecting to is working ok. I still cannot see any text in the controler log file.

I started up Openremote without having my device ready to responde to telnet requests and I can see error messages

in the console window cannot extablish gateway connection. Do I have to have the device powered up and ready to receive telnet

messages before I start OpenRemote.

Pierce

Posted by pjmm at Mar 29, 2011 20:48

Hi Pierce,

The fact you are getting the cannot establish connection messages shows that the gateway code is working.

You do not need your telnet server running before you start the controller, if you start it after you start the controller the gateway manager will re-establish the connection when your server is ready.

I'm not sure what you mean in your previous post:

Would it be possable to have a command to disconnect the telnet sesion as this would begood to

When/why would you want it to disconnect?

Rich

Posted by kurrazyman at Mar 29, 2011 20:56

Pierce,

I have a lot on this evening so will hopefully be able to provide you with some information about scripting tomorrow.

Rich

Posted by kurrazyman at Mar 29, 2011 21:13

Hi Richard

The way I would see the remote working would be you would scrole to a screan on your remote

The first button you would press would open the Telnet connection you would then press other buttons to

turn up Volume etc and when you nove to another page the telnet session would close.

Does that make sence

Pierce

Posted by pjmm at Mar 29, 2011 21:29

Hi Richard

could you give me an example of how I can send multible commands in the one string

Pierce

Posted by pjmm at Apr 01, 2011 17:33

Hi Pierce,

To send multiple commands you can either use the existing command syntax: -

<command id="1" protocol="telnet">
<property name="port" value="1234" />
<property name="command" value="null|command1|null|command2|null|command3" />
<property name="ipAddress" value="0.0.0.0" />
</command>

The null part of the command is the waitForString (my gateway code doesn't use this).

Or you can use the new gateway syntax (this isn't finalised but give you a bit more control: -

<command id="1" protocol="telnet">
<property name="port" value="1234" />
<property name="ipAddress" value="0.0.0.0" />
<property name="send" value="command1" />
<property name="send" value="command2" />
<property name="send" value="command3" />
<property name="read" value="" />
</command>

The read property at the end is needed for commands that are used by sensors (to read the response from the server), if you don't add the read property the gateway will insert one for you automatically.

Apologies for not getting back about the scripting, had a busy week and Juha is making some big changes to the controller at the moment so I want to try and work with him and develop a unified scripting API. Will keep you posted.

Rich

Posted by kurrazyman at Apr 01, 2011 17:41

Hi Richard

will designer create the xml for multible commands for your interface or will it have to be xml have to be edited manually

Pierce

Posted by pjmm at Apr 01, 2011 17:57

The designer can't generate the new command xml format so if you want to use the designer just use the old command format:

<command id="1" protocol="telnet">
<property name="port" value="1234" />
<property name="command" value="null|command1|null|command2|null|command3" />
<property name="ipAddress" value="0.0.0.0" />
</command>

Posted by kurrazyman at Apr 01, 2011 18:07

Hi

should the second example work with your new version of telnet?

I tried using the new format but it did not work

Pierce

Posted by pjmm at Apr 01, 2011 18:11

It appears that I broke the support for the new command format when I added support for the old format; I have now fixed this and uploaded the fix to the repository.

Posted by kurrazyman at Apr 01, 2011 18:31

Hi Richard

many thanks for the quick fix it woorks great.

Have you any time lines for when the scripting might be available

Pierce

Posted by pjmm at Apr 04, 2011 19:10

Glad it is now working for you.

I will try and get some information from Juha about the work he is doing as I'm hoping we can link my command scripting with his event scripting plans (should be lots of exciting functionality in the next release of OpenRemote).

Rich

Posted by kurrazyman at Apr 04, 2011 20:27

Hi Richard

many thanks for the update. I have one further question should I be able to
see the reply in OpenRemote from what I am telnetting to. I know it is sending
a reply because I tested it using Putty.
I have inserted <property name="read" value="" />

I have looked in the controller log but I cannot see the reply.

Am I looking in the correct place

Pierce

Posted by pjmm at Apr 04, 2011 22:47

You won't see the response from the server in the log file; generally only errors/warnings are written there. If all the server responses were added to the log file they would grow very quickly!

The only place to see the response is to use it in a sensor (custom sensor for string responses), you can then use the REST API to see the response in a web browser.

Rich

Posted by kurrazyman at Apr 05, 2011 07:17

Hi, I have just discovered this project. This branch looks like a very interesting feature. I have a Pioneer SC-LX82 that requires telnet so would like to help test this out.

I have tried to build your feature/Controller_2_0_0_Alpha11_Manager/ branch but seem to be missing GatewayCreatorUtil

javac /Users/matbic/openremote/OpenRemote_Boss_2_0_0_Alpha8/Controller_2_0_0_Alpha11/src/org/openremote/controller/service/impl/ControllerXMLChangeServiceImpl.java:45: cannot find symbol
javac symbol : class GatewayCreatorUtil
javac location: package org.openremote.controller.utils
javac import org.openremote.controller.utils.GatewayCreatorUtil;

I think it is missing from the repository? Or have I misunderstood something? Thanks.

Posted by mandp at May 24, 2011 20:08

Hi Mat,

You've done nothing wrong, I cleaned up some of the code and hadn't fully committed it.....should work now if you checkout again.

Haven't worked on this in some time but should all be working, the information in this thread should explain the basics.

Rich

Posted by kurrazyman at May 24, 2011 20:44

Thanks for the quick reply. It now builds without errors. However on starting tomcat I get a:

SEVERE: Error listenerStart

and then /controller startup fails.

Any suggestions on how to fix this? I suspect we are still missing some things from the repository (in controller/gateway/protocol there is no http directory) because in the logs I find:

May 24, 2011 11:12:04 PM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Error configuring application listener of class org.openremote.controller.statuscache.InitCachedStatusDBListener
org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class org.openremote.controller.gateway.protocol.http.HttpProtocolBuilder for bean with name 'httpProtocolBuilder' defined in class path resource applicationContext.xml; nested exception is java.lang.ClassNotFoundException: org.openremote.controller.gateway.protocol.http.HttpProtocolBuilder
Caused by: java.lang.ClassNotFoundException: org.openremote.controller.gateway.protocol.http.HttpProtocolBuilder
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1387)

...

May 24, 2011 11:12:04 PM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Error configuring application listener of class org.openremote.controller.gateway.InitConnectionManager
org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class org.openremote.controller.gateway.protocol.http.HttpProtocolBuilder for bean with name 'httpProtocolBuilder' defined in class path resource applicationContext.xml; nested exception is java.lang.ClassNotFoundException: org.openremote.controller.gateway.protocol.http.HttpProtocolBuilder
Caused by: java.lang.ClassNotFoundException: org.openremote.controller.gateway.protocol.http.HttpProtocolBuilder
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1387)

...

Posted by mandp at May 24, 2011 23:23

Hi Mat,

There is definitely a http folder in the protocol package so I'm not sure about your checked out files; I have run the Manager branch on my PC and it works OK.

Rich

Posted by kurrazyman at May 25, 2011 17:05

Hi,

I have just checked everything through, did a fresh check out etc. but still same error.

In ^/branches/feature/Controller_2_0_0_Alpha11_Manager
on http://openremote.svn.sourceforge.net/svnroot/openremote

The only http folder I can see is in ... src/org/openremote/controller/protocol

But, the error message is looking for one in ... src/org/openremote/controller/gateway/protocol

and I don't see one there.

Or again am I wrong. Thanks for the help.

Posted by mandp at May 25, 2011 18:09

Sorry Mat, would help if I read things properly wouldn't it!

You're spot on about the missing files, I have now rectified the problem...honest...bit of an SVN glitch.

Basically I have implemented the telnet, udp and http protocols using this connection manager branch, other protocols will revert to using the old (un-managed) mechanism for connection. Hence why there are protocols inside the gateway package.

Rich

Posted by kurrazyman at May 25, 2011 20:02

Great the server starts up! Now for the fun bit - setting it all up! Thanks so much for this wonderful project.

Posted by mandp at May 25, 2011 23:29

I have found a small bug, do you want me to send you a patch file?

Works much better now!


Index: Gateway.java
===================================================================
--- Gateway.java	(revision 4403)
+++ Gateway.java	(working copy)
@@ -372,7 +372,7 @@
             this.connectionState = EnumGatewayConnectionState.CONNECTED;
             
             // Disconnect unless connection type is permanent
-            if (this.connectionType != EnumGatewayConnectionType.PERMANENT || this.connectionType != EnumGatewayConnectionType.TIMED) {
+            if (this.connectionType != EnumGatewayConnectionType.PERMANENT && this.connectionType != EnumGatewayConnectionType.TIMED) {
                protocolDisconnect();
             }
          } catch (GatewayConnectionException e) {

Posted by mandp at May 30, 2011 15:00

Hi Mat,

Well spotted, unfortunately I was making some changes to this code a couple of months ago and then I discovered that Juha is/has been working on some fundamental changes to the controller and so I dropped what I was doing until I had an understanding of where things were heading long term.

As such I was 'mid flow' with the work on the Connection Manager when I stopped work on it and I imagine there are a few bugs in there as you've already discovered one. I'll add you're change to SVN though.

Rich

Posted by kurrazyman at May 30, 2011 15:32

That's OK.

Can you give me some pointers on how to set up a script. Reading through the source code is quite slow!

I think I have figured out the following:

  1. Create a JavaScript file (I see you send in statusCache and the args, but have not jet figured out how to manipulate them in the script, if you had an example that would be really very helpful)
  2. Put the JavaScript file in a scripts directory under controller.xml
  3. Edit controller.xml (not sure what exactly to do here)

Any example would be great. If you like I can write up the documentation to help others.

Thanks

Posted by mandp at May 30, 2011 16:29

Hi Mat,

It's all very experimental stuff that I developed to meet my needs to interface with PulseAudio and MPD on Linux and the scripting gives you a lot of power but due to the experimental nature there is no standard on what controller objects are accessible from inside the scripts but if you have a need for this and would like to try it out the following will help: -

Script Locations - As you say they go in a folder called scripts in the Tomcat controller webapps folder (depends on your CATALINA_HOME environment variable definition, but this will be the same folder that the controller.xml and panel.xml files are loaded from). At present JavaScript is supported and script files should be saved with a .js extension.

Usage - As you've spotted the statusCache instance is passed into the script as well as the Key Value pairs (args) the args come from the command definition in the controller.xml. An example script command definition and the script that it uses is shown below: -

Controller.xml Command Definition

   <command id="858" protocol="telnet">
      <property name="port" value="6602" />
      <property name="host" value="192.168.1.71" />
      <property name="send" value="command=status" />
      <property name="read" value="" />
      <property name="script" value="scriptName=getStringAfterWord.js;word=volume" />
    </command>

Script (getStringAfterWord.js)

function run(currentValue) {
   var result = "";
   if (word != null) {
      var re = null;
      eval("re = new RegExp(/^.*" + word + ".\\s(\\-*\[\\d|\\w]+).*$/im);");
      if (re != null) {
         var match = re.exec(currentValue);
         if (match) {
            result = match[1];
         }
      }
   }
   return result;
}

The script must contain a function called run which takes the current command value as its' only argument (this is the value returned from the previous command action, should be a read action). This script applies a regex to a load of text received from a telnet server after issuing the command 'status', the argument 'word' that is defined in the controller.xml allows me to re-use the script depending on what I want to filter out.

To explain something, sending the statusCache instance into the script was a kludge on my part because in my environment I have a sensor that reads a load of data from a telnet server (PulseAudio) and the command returns 33k of text, as you can imagine it slows down the controller somewhat (especially as I have 10 sensors that issue the same command!); I added pollingInterval control to the connection Manager so each command can be customised to only run every n Seconds but this wasn't enough on this occasion because my sensor was connected to a toggle button so by increasing the polling interval of the sensor command what happened was that when I clicked the button it's state wouldn't update until the next time the sensor command was executed (this caused big delays and caused my panel to not be in sync with what was actually happening). So I decided to kludge it by updating the statusCache as part of my button action command and I set the sensor command to a pollingInterval of 60 Seconds, this meant that when I clicked the button on the panel its' state would update instantly but just in case things lost sync (very unlikely but just in case) the sensor command once a minute actually queries the telnet server and retrieves the state.

There was an issue with this, I couldn't find a way of easily determining the sensor ID that the toggle button was linked to when the 'onPress' button command was being executed. So I ended up feeding in the sensor ID as an argument which is no good long term because as soon as the controller.xml changes the sensor ID would need to be altered to match. The work Juha is doing on the controller will help resolve this long term but this kludge works for me in the meantime. Below is a controller.xml command and a script demonstrating this statusCache kludge.

Controller.xml Command Definition

    <command id="854" protocol="telnet">
      <property name="port" value="5050" />
      <property name="host" value="192.168.1.71" />
      <property name="send" value="command=set-sink-input-volume 22 ${param}" />
      <property name="script" value="scriptName=SetSensorValue.js;setValue=${param};sensorId=755;" />
    </command>

Script (SetSensorValue.js)

function run(currentValue) {
   if(setValue != null && sensorId != null) {
      statusCache.setValue(sensorId, setValue);
   }
   return null;
}

Hope this helps,

Rich

Posted by kurrazyman at May 30, 2011 17:12

Hi Mat

if you were to documentation how this works it would be a great help as I would
also like to get this feature to work

Pierce

Posted by pjmm at May 30, 2011 18:24

Hi Rich,

What's the status with this branch? I've basically written a TCPProtocol.java imp for you to add (another C/P of your work on telenet but saves you the bother)
It compiles fine, but I'm unable to test it as following the gateway layout described in your below posts, it results in the following log error:

Controller2011-06-25 12:32:49,503 WARN main org.openremote.controller.service.impl.GatewayManagerServiceImpl.initGatewaysWithControllerXML(107) | Error retrieving gateway elements from controller.xml.

Looking at the GatewayBuilder.java code, it looks like your intention is/was to define a gateway tag in the controller.xml

  • <pre>
    Unknown macro: {@code * <gateway protocol = "protocol-id" > * <property name = "argname1" value = "..."/> * <property name = "argname2" value = "..."/> * <property name = "argnameX" value = "..."/> * </gateway> * }
    </pre>

Only this isn't defined in the xml schema so throws a SAX parser error.... I'm confused about how the controller.xml should now look.

I'd really like to test this TCPProtocol imp to confirm it works and add it to your branch for all to benefit from, that is if the branch is still active.

Phill

Posted by flipit at Jun 25, 2011 11:44

Hi Phil,

Apologies for not replying to your question the other day about the telnet branch location; as you discovered it has been merged back into the trunk and is now defacto for everyone.

With this manager branch yes I did intend for a different xml schema but it also works with the old/current xml schema for backwards compatibility. There are a couple of classes in the utils package that try to convert the current xml controller.xml into the format I intended for this branch (but this was something I bolted on last minute so users could use this branch without having to alter the controller.xml and so it isn't as robust as I'd like).

I assume the work you have done on the TCP protocol read method is along the same lines as what I did in the telnet branch, in this case your work can be put into the existing trunk (if you email your two .java files to Juha or even ask him by mail where you can commit these files to he can help you out).

As you say it would be great if everyone can benefit from your efforts.

Hope this helps,

Rich

Posted by kurrazyman at Jun 25, 2011 13:06

Hi Richard

some time ago you were able to help me with multible Telnet commands and I used

<command id="1" protocol="telnet">
<property name="port" value="1234" />
<property name="ipAddress" value="0.0.0.0" />
<property name="send" value="command1" />
<property name="send" value="command2" />
<property name="send" value="command3" />
<property name="read" value="" />
</command>

Is this now merged into the main branch

Regards

Pierce

Posted by pjmm at Jun 25, 2011 15:30

Hi Pierce,

The multiple command actions within a single command is something I created in this Manager branch only; it is not available in the main version unfortunately.

Rich

Posted by kurrazyman at Jun 25, 2011 16:22

Hi Rich,

Ok, I patched GatewayManagerServiceImpl.java & ApplicationContext.xml, and created SocketProtocol.java & SocketProtocolBuilder.java based on your version of the telnetprotocol.

All appears to be working fine. Not fully tested, but certainly updated my basic panel.

Patches against svn diff sent to Juha for inclusion in the branch.

Thanks

Phill

Posted by flipit at Jun 25, 2011 17:37

Hi Phill,

Once again well done for taking the initiative on this.

If the work you have done is for the Manager branch (i.e. the protocol changes you made reside in the controller.gateway.protocol.tcp package rather than the controller.protocol.tcp package) then it will only be compatible with this Manager branch if this is the case then I can commit your changes to this branch if you wish.

Rich

Posted by kurrazyman at Jun 25, 2011 18:42

Hi Rich, Yes this patch is for your Manager branch (I got carried away with socket code and updated both the trunk and your manager branch...)

How can I send you the patch ?

Phill

Posted by flipit at Jun 25, 2011 18:56

That's good work, there isn't that much difference between the code you need in each protocol implementation so well done for sorting them both. You can either send me a diff of your changes to the Manager branch or send me the SocketProtocol.java & SocketProtocolBuilder.java files and I will commit them to the branch.

Rich

Posted by kurrazyman at Jun 25, 2011 19:56

Hi Richard

Using the latest release and telnet when I send a command the sensor send a string for on or off
ie 0021003000380040 OHCAObjectFlag.Get-1 is on
0021003000380040 OHCAObjectFlag.Get-0 is Off

How can I read this do I put the string in Default Read Response: or do I have to use Read Regex Filter:
What should I enter under Read Regex Group:

Do I have to make any manual changes to the XML

Any help would be appriciated

Pierce

Posted by pjmm at Jun 28, 2011 22:23

Hi Pierce,

You can use the following (I haven't tested this but should work):-

Read Regex Filter: ^(?i)(?s)(?m).*get.(\d)$
Read Regex Group: 1
Default Read Response: 0 (This is the value the read command returns if no match can be found)

The above will cause the read command to return a 0 or 1 value.

Rich

Posted by kurrazyman at Jun 29, 2011 19:28

Hi Richard

many thanks for that info.

For aswitch I will need Yes or No how can I translate 0 to No and 1 to Yes

Many thanks

Pierce

Posted by pjmm at Jun 29, 2011 19:36

Hi Pierce,

For switch sensors you need an 'on' or 'off' response I believe, if you're lucky it might accept the 0 or 1 (give it a try), if it does not then there is no way to convert the value using the standard controller, you would have to use my connection manager branch and use a javaScript to do the conversion.

Rich

Posted by kurrazyman at Jun 29, 2011 19:44

Hi Richard

I tried ^(?i)(?s)(?m).*get.(\d)$ in Regex and it did not work

Can you please look at ist again

Pierce

Posted by pjmm at Jun 29, 2011 20:05

Hi Richard

in my example I had a "-" Get and 1

it should read 0021003000380040 OHCAObjectFlag.Get1

Can you tell me what I should change

Pierce

Posted by pjmm at Jun 29, 2011 20:12

The regex I gave you should work for either a "-" or a "." you could try the below to make the regex more flexible (I tried this using an online regex tester and it works ok):

^(?i)(?s)(?m).get.(\d).$

How do you know it is not working; have you tried using the command as a source for a custom sensor and then using the custom sensor in a label on your panel (you will see the value that the read command returns then displayed on your panel).

Rich

Posted by kurrazyman at Jun 29, 2011 20:54

Hi Richard

I have set it up as you describe using the command as a source for a custom sensor and then using the custom sensor in a label on your panel (you will see the value that the read command returns then displayed on your panel).

I am not getting the desplay to change with a "0" or "1"

It does not change.

Either it is not getting passed on or OP wont work with 0 and 1

Is there any way to log what if it is sending 1 or 0

Regards

Pierce

Posted by pjmm at Jun 29, 2011 21:32

Hi Pierce,

Sorry I noticed that the regex I gave you in my previous post has been changed by the wiki engine:

^(?i)(?s)(?m).*get.(\d).*$

Try that

Posted by kurrazyman at Jun 29, 2011 22:02

Hi Richard

I have now changed the responce I get for on and off and I would be greatfull
if you could give me the correct regex for the following

0021003000380048 0HCAObjectFlag.GetButtonAOff
0021003000380048 0HCAObjectFlag.GetButtonAOn

I believe if you can give me this I should have it working

Many thanks

Pierce

Posted by pjmm at Jun 30, 2011 21:07

Hi Pierce,

^(?i)(?s)(?m).*getbuttona(\w{2,3}).*$

Tested at: -

http://www.regexplanet.com/simple/index.html

Useful guide for learning Regex:

http://www.regular-expressions.info/reference.html

Rich

Posted by kurrazyman at Jun 30, 2011 22:24

Hi Richard

many thanks for the string I tested it and it gives on or off depending on the
different strings.

I have put the string in under "Read Regex Filter:"
I have entered 1 ubder "Read Regex Group:"
and 0 under "Default Read Response:"

It still does not change the state of the switich

I can see the string with no errors in the controler log

Have you any other ideas as to what I might be dooing wrong

Many thanks

Pierce

Posted by pjmm at Jul 01, 2011 01:13

Hi Pierce,

I'm not sure without looking at the source code but maybe it's because your telnet responses have a capital letter at the start (i.e. On insead of on, Off instead of Off). Maybe the sensor is case sensitive.

Rich

Posted by kurrazyman at Jul 01, 2011 07:04

Hi I have tested On and on

I see the following error in controler log

Controller2011-07-01 12:53:11,738 ERROR HTTP\-Thread\-2 org.openremote.controller.protocol.telnet.TelnetCommand.setResponseFilterGroup(218) | filter group property in controller.xml is not an integer
java.lang.NumberFormatException: For input string: "^(?i)(?s)(?m).getbuttona(\w

Unknown macro: {2,3}
).$"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)

Can you tell me what this refers to

Pierce

Posted by pjmm at Jul 01, 2011 13:18

The log message suggests that you have put the regex in the Response Filter Group?

Posted by kurrazyman at Jul 01, 2011 14:56

Hi Richard

tis is the xml can you see where I am going wrong

Pierce

<?xml version="1.0" encoding="UTF-8"?>
<openremote xmlns="http://www.openremote.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openremote.org http://www.openremote.org/schemas/controller.xsd">
<components>
<switch id="62">
<on>
<include type="command" ref="64" />
</on>
<off>
<include type="command" ref="64" />
</off>
<include type="sensor" ref="63" />
</switch>
</components>
<sensors>
<sensor id="63" name="Button A" type="switch">
<include type="command" ref="66" />
<state name="on" />
<state name="off" />
</sensor>
</sensors>
<commands>
<command id="64" protocol="telnet">
<property name="port" value="2000" />
<property name="statusDefault" value="1" />
<property name="command" value="null|HCA0002010000001|null|002500350051 HCAObjectProgram.OnRemote - ButtonA|null|00180027 HCAAppTerminate|" />
<property name="statusFilter" value="1" />
<property name="statusFilterGroup" value="1" />
<property name="timeout" value="99" />
<property name="ipAddress" value="192.168.1.7" />
</command>
<command id="66" protocol="telnet">
<property name="port" value="2000" />
<property name="statusDefault" value="0" />
<property name="command" value="null|HCA0002010000001|null|002500330040 HCAObjectFlag.GetButtonA|" />
<property name="statusFilter" value="^(?i)(?s)(?m).getbuttona(\w

Unknown macro: {2,3}
).$" />
<property name="statusFilterGroup" value="1" />
<property name="timeout" value="99" />
<property name="ipAddress" value="192.168.1.7" />
</command>
</commands>
<config>
<property name="controller.roundrobin.tcpserver.port" value="20000" />
<property name="multicast.address" value="224.0.1.100" />
<property name="controller.roundrobin.multicast.port" value="10000" />
<property name="multicast.port" value="3333" />
<property name="controller.groupname" value="floor20" />
<property name="Macro.IR.Execution.Delay" value="500" />
<property name="controller.roundrobin.multicast.address" value="224.0.1.200" />
<property name="lircd.conf.path" value="/etc/lircd.conf" />
<property name="controller.groupmember.autodetect.on" value="true" />
<property name="webapp.port" value="8080" />
<property name="copy.lircd.conf.on" value="true" />
<property name="irsend.path" value="/usr/local/bin/irsend" />
<property name="resource.upload.enable" value="true" />
<property name="controller.applicationname" value="controller" />
</config>
</openremote>

Posted by pjmm at Jul 01, 2011 17:35

Hi Pierce,

When pasting code into your message you need to use the following braces around the code:

{code}

{code}

I can't see what would have caused the error you were getting but for all of your commands, in the command property I don't think you want the '|' at the end and for the switch sensor command the statusDefault property should 'on' or 'off'.

For example: -

<command id="66" protocol="telnet">
<property name="port" value="2000" />
<property name="statusDefault" value="off" />
<property name="command" value="null|HCA0002010000001|null|002500330040 HCAObjectFlag.GetButtonA" />
<property name="statusFilter" value="^(?i)(?s)(?m).getbuttona(\w{2,3}).$" />
<property name="statusFilterGroup" value="1" />
<property name="timeout" value="99" />
<property name="ipAddress" value="192.168.1.7" />
</command>

As I don't have a telnet server that produces the same output as yours there's only so much I can do my end. Have you tried my suggestion from before of creating a custom sensor and using it to set the value of a label on your panel (or you could interrogate the sensor value using the REST API), you should see either 'on' or 'off' being displayed.

Rich

Posted by kurrazyman at Jul 01, 2011 20:52

Hi Richard

I have set up 2 sensors but the status does not change. i am receiving the correct string
This is an extract from the Controler Log.

Pierce

Controller2011-07-01 22:08:32,629 INFO Thread\-3 org.openremote.controller.protocol.telnet.TelnetCommand.readString(349) | received: HCA0202010003074
Controller2011-07-01 22:08:32,629 INFO Thread\-3 org.openremote.controller.protocol.telnet.TelnetCommand.sendString(327) | send: 002500330040 HCAObjectFlag.GetButtonA
Controller2011-07-01 22:08:32,629 INFO Thread\-4 org.openremote.controller.protocol.telnet.TelnetCommand.readString(349) | received: HCA0302010003074
Controller2011-07-01 22:08:32,629 INFO Thread\-4 org.openremote.controller.protocol.telnet.TelnetCommand.sendString(327) | send: 002500330040 HCAObjectFlag.GetButtonB
Controller2011-07-01 22:08:33,629 INFO Thread\-3 org.openremote.controller.protocol.telnet.TelnetCommand.readString(349) | received: 0021003000380048 0HCAObjectFlag.GetButtonAoff
Controller2011-07-01 22:08:33,629 INFO Thread\-4 org.openremote.controller.protocol.telnet.TelnetCommand.readString(349) | received: 0021003000380048 0HCAObjectFlag.GetButtonBoff
Controller2011-07-01 22:08:33,629 INFO Thread\-4 org.openremote.controller.protocol.telnet.TelnetCommand.read(372) | Telnet Read Status: No Match using Regex: '^(?i)(?s)(?m).getbuttona(\w

Unknown macro: {2,3}
).$' on response from command 'null|HCA0002010000001|null|002500330040 HCAObjectFlag.GetButtonB|'

Posted by pjmm at Jul 01, 2011 22:17

Hi Richard

I just spotted why it was giving that error the string need b instead of a

So now I know that the string is received and read why is it not updating the display?

Pierce

Posted by pjmm at Jul 01, 2011 22:34

It could be a bug with your designer account (I have had this happen to me); this causes the panel.xml and controller.xml to lose sync.

I would suggest quickly creating another online designer account and quickly add a custom sensor with your read command and use a label to see if you can see the value.

If this works then Juha may be able to help sort your designer account out or you'd have to delete everything and start again (depends how many panels, sceens, etc you have defined).

Rich

P.S: Still don't think you should have the '|' at the end of your 'null|HCA0002010000001|null|002500330040 HCAObjectFlag.GetButtonB|' command (just in case it causes problems).

Posted by kurrazyman at Jul 01, 2011 22:46

Hi Richard

I tried another account but still the same

Pierce

Posted by pjmm at Jul 01, 2011 23:45

Hi Pierce,

You said above:

I have set up 2 sensors but the status does not change. i am receiving the correct string

Does that mean when you tried using a label on your panel and setting the label command to one of your read commands the label displayed either 'on' or 'off' on the panel when it was running?

If the answer is yes then there is no reason why the switch shouldn't update if the answer is no then your problem must be in the command itself. In the console window if the regex fails to find a match you will see messages like below:

[Controller] 2011-07-02 08:45:33,022 ERROR [Thread-1870] org.openremote.controller.protocol.telnet.TelnetCommand.read(372) | Telnet Read Status: No Match using Regex: '^(?i)(?m)(?s).*volume.\s(\d+).*$' on response from command 'OK|status'

Do you see any messages like that in the console window?

What code branch and revision are you running?

Rich

Posted by kurrazyman at Jul 02, 2011 08:46

Hi Richard

Just to confirm I have set up a label and the status of this does not change.

I am receiving no errors on the Console Window or in the Controler Log file.

I have changed the data string which is returned to OpenRemote when it asks for
the status. I now get No Match using Regex in the Controler log file and no
errors in the Console Window.

I believe I am receiving back the correct string and Regex is getting a match
and nothing happins.

I appriciate all your help on this

Pierce

Posted by pjmm at Jul 02, 2011 15:27

Hi Pierce,

I would recommend querying the controller directly using the REST XML API: -

REST XML API

So assuming your controller is running on the PC you are using you'd type in: -

http://localhost:8080/controller/rest/status/{sensor_id}

Replace {sensor_id} with the switch or custom sensor id from your controller.xml

See what gets returned.

Rich

Posted by kurrazyman at Jul 02, 2011 16:15

Hi Richart

This is my XML
<components>
<button id="70">
<include type="command" ref="72" />
</button>
<label id="68">
<include type="sensor" ref="71" />
</label>
</components>
<sensors>
<sensor id="71" name="Button A" type="switch">
<include type="command" ref="73" />
<state name="on" />
<state name="off" />
</sensor>

I tried Unknown macro: {sensor_71} ">http://localhost:8080/controller/rest/status/

Unknown macro: {switch_71} ">http://localhost:8080/controller/rest/status/
Unknown macro: {switch_71}
Unknown macro: {71} ">http://localhost:8080/controller/rest/status/
Unknown macro: {71}

All returned the same message
<message>No such sensor whose id is :%7Bswitch_71%7D</message>

Pierce

Posted by pjmm at Jul 02, 2011 17:03

The URL should be:

http://localhost:8080/controller/rest/status/71

Not:

http://localhost:8080/controller/rest/status/{sensor_71}

Or:

http://localhost:8080/controller/rest/status/{switch_71}

Or:

http://localhost:8080/controller/rest/status/{71}

Rich

Posted by kurrazyman at Jul 02, 2011 17:08

Hi Richard

http://localhost:8080/controller/rest/status/71

<status id="71">off</status>

The string which is being received is
0021003000380047 0HCAObjectFlag.GetButtonaon
this should make the Sensor On.

I also sent an Off commang string and still got
<status id="71">off</status>

Pierce

Posted by pjmm at Jul 02, 2011 17:22

Edit your controller.xml and set the default read response to "on" then query the REST API again: -

http://localhost:8080/controller/rest/status/71

If you see:

<status id="71">on</status>

Then you are likely not getting a regex match, The response from your Telnet server that you wrote above:

0021003000380047 0HCAObjectFlag.GetButtonaon

Is different to what you said before (before there was a '.' before the 'on') this makes a difference to your regex so you need to be exact.

Rich

Posted by kurrazyman at Jul 02, 2011 17:29

Hi Richard

no change still responds with <status id="71">off</status>

Regarding the change in the string I changed "A" to "a" to see if it made any difference.

I tested both and both work OK giving the corredt result

Pierce

Posted by pjmm at Jul 02, 2011 17:46

The regex I gave you is case insensitive (for some reason I thought you had a '.' after the getbuttona).

If you are still getting an 'off' response then it must mean that the regex is finding a matching 'off' at the end of the response from the server.

Try removing the regex stuff from your command and query the REST API again just to be sure you are getting on and off responses from your telnet server when you switch on/off the device being queried.

Something like:

<command id="71" protocol="telnet">
<property name="port" value="2000" />
<property name="command" value="null|HCA0002010000001|null|002500330040 HCAObjectFlag.GetButtonA" />
<property name="ipAddress" value="192.168.1.7" />
</command>
Posted by kurrazyman at Jul 02, 2011 17:54

Hi Richard

We have set the default setting to On and after your last mail I changed the string which is being sent to
Openremote Regex was unable to decode it and now when I send http://localhost:8080/controller/rest/status/71

I get <status id="71">on</status>.

So It would appear that the Regex is decoding both the on and off message as off.

I tested both messages and I received the correct responce On and Off.

0021003000380047 0HCAObjectFlag.GetButtonaon
"for On"

0021003000380048 0HCAObjectFlag.GetButtonaoff
"for Off"

I have re tested these and the responce is correct.
I also tested using "A" and they also gave the correct answer

Pierce

Posted by pjmm at Jul 02, 2011 18:47

Hi Pierce,

I don't understand how the regex can be returning off even when the response from your server is '0021003000380047 0HCAObjectFlag.GetButtonaon' because the string doesn't even contain the characters 'off'.

I have tested the below regex and it should work for buttonA and buttonB and will return 'on' or 'off' if the response format is correct.

^(?i)(?s)(?m).*getbutton.(\w{2,3}).*$

Rich

Posted by kurrazyman at Jul 03, 2011 10:06

Hi Richard

I have tested this new string and it gives the same results as the last one

For some reason the Regex is sending "off" each time

How can we proceed now

Pierce

Posted by pjmm at Jul 03, 2011 13:06

Hi Pierce,

Can you confirm which revision of the controller you have checked out of the repository so I can make sure I am running the same.

The only other thing I would suggest is use the remote debugging within Eclipse so you can see exactly why 'off' is always being returned.

Rich

Posted by kurrazyman at Jul 04, 2011 17:19

Hi Richard

the version I am currently running is OpenRemote-Controller-2.0.0_SNAPSHOT_20110611\bin

I have not used Eclipse before can you point me in the right direction

Pierce

Posted by pjmm at Jul 04, 2011 21:16

Hi Pierce,

I shall try that branch tomorrow and see what I get with my telnet servers.

I'll see if the branch works OK for me before you need to go down the eclipse debugging route.

Rich

Posted by kurrazyman at Jul 04, 2011 22:08

Hi Pierce,

I have downloaded the branch you are using and it works OK for me so all I can suggest is that there is something strange with your setup and the only option you would have is to go down the debugging route: -

  • Install Eclipse (can download off the net...it's free)
  • Import the controller source code as an existing project
  • Start the apache server using the jpda run arguments ('openremote.bat jdpa run') this puts it into debug mode
  • Create a remote java application debugging configuration in Eclipse, the defaults should be ok (change host to PC running controller if not localhost)
  • You should then be able to specify break points in the source code within Eclipse and you'll be able to see all of the controller variables

Rich

Posted by kurrazyman at Jul 06, 2011 17:27

Hi Richard,

what is the status of this branch?
I just checked-out and compiled but I don't see where you make the link between the "regular" telnet command and the "protocol" telnet command.
It looks like your gateway stuff is not touched if I deploy regular setup from the designer.
You are implementing a ProtocolBuilder but normally a CommandBuilder is used.

Thanks,
--Marcus

Posted by mredeker at Jul 31, 2012 20:54

Hey Marcus,

Only just saw your question (don't think I got an email to let me know for some reason but just happened to look through the Design Forum).

I haven't looked at this for over a year (the Web Console has been keeping me occupied) but I'll try and jog my memory.

All the code I created in this branch sits inside the Gateway package to avoid conflicts with existing code and it was initiated by my own needs to have connection management specifically for Telnet but more generally also.

I only implemented the Telnet, UDP, HTTP and TCP protocols into the gateway code so any other protocols will behave as normal.

Any commands that use the four implemented protocols should use the connection manager code, and this should be apparent by the messages produced in the console window. A thread is created for each server and protocol (I call these gateways) and then all commands that use that gateway are associated with it and the Gateway Manager deals with command requests and connecting/disconnecting from the server.

Regarding the ProtocolBuilder, I wanted to get away from the idea of the Commands containing protocol related information, hence the extra layer of abstraction, to deal with this and to work with the existing controller.xml schema the gateway manager re-writes the XML on the fly to add in new gateway and protocol elements (have a look at org.openremote.controller.service.impl.GatewayManagerServiceImpl.java for details of what is going on).

Not sure why you don't see anything happening but I will fire it up my end tomorrow evening and see what is going on.

Rich

into the connection management (gateway) code and yes you are right that if you supply in a standard controller.xml from the designer my gateway code won't do much and the standard mechanisms will do the work (this was all to allow my code to co-exist with the current code).

Posted by kurrazyman at Sep 05, 2012 21:59

Hi Rich,
thanks for the answer. Do you have an example of a controller.xml where your code is used?
--Marcus

Posted by mredeker at Sep 05, 2012 22:17
Document generated by Confluence on Jun 05, 2016 09:29