This page last changed on Jul 04, 2013 by raphyot.

Hi everyone,

I'm looking at OR for a few days now, and I must say I really love it!
I have several home made arduino sending xPL, unfortunately it seems this can't work with OR at the moment. I'm ok to now follow the full xPL implementation (keepalive etc...), but I'd need at least to listen to UDP messages, and be able to send some. And I have problems with both...

1 Listening UDP:

It seems a common issue while looking at the feedback, using UDPListener with rules is not really suitable for a fully UDP based protocol. I need UDPListener to send the match back to a custom sensor. Hence, I made the following change that works fine for me:

diff UDPListenerCommand.java 

114,118c114
<                     if (matcher.groupCount()>0)  {
<                       sensor.update(matcher.group(1));
<                     } else {
<                       sensor.update(""+System.currentTimeMillis());
<                     }
---
>                     sensor.update(""+System.currentTimeMillis());

With this code, if you add a regex in UDPListener without the group match '(' and ')', it will work as before.
If you add a group match, the first match will be send as value (not sure this is best practice in OR?). I use the following regex to match an xPL message:

(?s).*arduino.rgb.*device=pbed.rgb.*command=setcolor.*color=(\w+).*

And this work fine. I didn't understand the developer doc (it seems outdated?) and struggled with subversion to find a good snapshot, I hope it's ok to show the diff in the forum.

2 Sending UDP:

The problem with sending UDP is that it seems I can't enter control character, which I'm using to parse the xPL payload on my arduino. XML should take to enter a new line character, however when I enter this in the UDP command it get translated as &#10; in the XML.
Not sure how to solve that one, anyway, I currently did a small python REST api to send xPL message with the HTTP command so I can keep playing with OR

Thanks, sending diff is fine.

For further development you can use the current snapshot tag in http://svn.code.sf.net/p/openremote/code/tags/snapshot/Controller_2_1_0_FM_SNAPSHOT-20130617/

Build with ant from top directory:

> ant

(obviously you need ant somewhere in your PATH first).

For further development and write access to Subversion, a contributor agreement is required. Please send me an email to juha at openremote dot org for details.

Posted by juha at Jul 05, 2013 09:08

Thanks Juha, that's the snapshot I worked with. Does that mean the diff proposal will be reviewed and integrated?
OR is already covering 99% of my needs so I'm not sure yet if I want a write access to Subversion
If getting a few bugs fixed through the forum works, I'm good with that.

Can someone still help on inserting a new line character from the designer (issue #2 above)?

Posted by raphyot at Jul 05, 2013 10:35

I'll review it but basically yes, will include. Have not decided on the version yet, may push it to 2.2, depends on the review. Will get back to it next week, going away for the weekend with no internet connection.

Posted by juha at Jul 05, 2013 10:51

Sorry for the delay, getting back to this now.

Posted by juha at Aug 02, 2013 21:53

Hello Raphael,

The binary for issue ORCJAVA-392 is available here OpenRemote-Controller-2.1.0_FM_ORCJAVA-392.zip and contains the regexp group modification to UDP listener. Currently scheduled for the 2.1 release branch.

Posted by juha at Aug 03, 2013 22:40

Raphael,

Did you find a solution to your line-feed issue, or are you still using the intermediate http scripts?

– Juha

Posted by juha at Aug 03, 2013 22:41

Thanks for adding the code!

No, I didn't find a solution for the line-feed issue, I'm still using an additional http script.

Posted by raphyot at Aug 06, 2013 12:04

Preparing patch for UDP linefeed (from Simon Vincent). Will get you a binary to test.

Posted by juha at Aug 06, 2013 12:08

Raphael,

I've deployed a test version of a UDP patch originally contributed by Simon. It required some modifications so would appreciate it if you have a chance to test it.

In https://composer.openremote.org/staging you can login and find in the UDP protocol a new configuration attribute 'line ending'. It should allow you to select CR, LF, or CRLF as option to include at the end of the UDP packet you send.

For the functionality to work, you will need to install a controller snapshot from today, available here: OpenRemote-Controller-2.1.0_FM_SNAPSHOT-2013-08-06.zip. It also includes the addition to UDP listener to accept a regexp group as input.

If you can provide feedback on whether these features work and we can push them forward to release.

Best regards,

– Juha

Posted by juha at Aug 06, 2013 16:26

Hi Juha,

I can test more over the weekend, but I just logged in the composer. From what I saw I can only enter one line in the "command" text field, and then chose how this line end.
What is needed with XPL is to have several lines ending by a carriage return, ideally using a textarea.

Thanks,
Raphael

Posted by raphyot at Aug 09, 2013 08:11

Hi - I'm using this snapshot but not having much luck. To test, I have a little python script which sends a UDP packet to port 8014. Using 'nc -ul 192.168.2.10 8014' I can verify the content arrives correctly to the specified port on the specified host. However, I can't seem to get OpenRemote (running on the same host) to react. I do see 2 messages on the console at startup which seem to be related:

DEBUG 2013-11-04 21:13:14,513 (XML): A switch sensor (Name = 'Fireplace light sensor', ID = 66) has an incomplete <state> element mapping, the value attribute is missing in <state name = on/>.
DEBUG 2013-11-04 21:13:14,513 (XML): A switch sensor (Name = 'Fireplace light sensor', ID = 66) has an incomplete <state> element mapping, the value attribute is missing in <state name = off/>.
}

This might explain the switch not reacting to the sensor. However, I would still have expected to see something show up in a log file somewhere. So, I guess I have 2 questions:

1. Should there be a log entry somewhere for UDP packets received on the UDPListner port?
2. What/where do I need to fix to resolve the 2 messages above?

My sensor is configured as UDPListner on port 8014, and the regexp is 'Fireplace_lt is: (\d)', in case that matters.

Thanks,
Jeff

Posted by kwazy at Nov 05, 2013 15:33

1. You will need to enable UDP listener DEBUG logging by adding the following into your Controller/webapps/controller/WEB-INF/classes/log4j.properties file:

log4j.logger.OpenRemote.Controller.protocol.udp_listener = ALL, udp-file
log4j.appender.udp-file = org.apache.log4j.RollingFileAppender
log4j.appender.udp-file.File = ../logs/udp/udp.log
log4j.appender.udp-file.Threshold = DEBUG
log4j.appender.udp-file.MaxFileSize = 500KB
log4j.appender.udp-file.layout = org.apache.log4j.PatternLayout
log4j.appender.udp-file.layout.ConversionPattern = %p %d (UDP): %m%n

2. The DEBUG statement is not an ERROR and nothing is required for you to do. It's just a note with regards to incomplete XML definitions the designer creates.

Posted by juha at Nov 05, 2013 22:42

Thanks for your reply Juha. I added the lines you listed to the file you specified, and I now see a udp.log in logs/udp. However, its size remains 0 when I test.

Additionally, I did some more playing around, including downgrading from openjdk 1.7 to openjdk 1.6 so that I could experiment with some rules. I've since commented out all rules, but I'm seeing the following console output when I send my test UDP packet, but only the first packet after a stop/restart:

ERROR [UDPListener Port: 8014]: Error in UDPListenerThread
java.lang.IllegalStateException: No match found
        at java.util.regex.Matcher.group(Matcher.java:485)
        at org.openremote.controller.protocol.socket.UDPListenerCommand$UDPListenerThread.run(UDPListenerCommand.java:122)
}

Thanks again - feel like I'm getting closer...

Jeff

Posted by kwazy at Nov 05, 2013 23:20

Hi Jeff,

Looks like an issue with the patch itself.

1) The IllegalStateException handling is outside of the listener loop so once it's thrown, the listener thread dies

2) The exception shouldn't happen in the first place, as it says it is caused by an illegal state, so a programming error

The related javadoc for the call causing the exception doesn't detail much:

IllegalStateException - If no match has yet been attempted, or if the previous match operation failed

But looking at the implementation after the patch above in this thread has been added, it does seem to be missing the relevant match() or find() call on the regexp matcher which might explain the "illegal state".

Any chance you'd be able to patch the patch yourself? The patch branch is in my workspace, and the relevant file is in http://svn.code.sf.net/p/openremote/code/workspace/juha/Controller_2_1_0_FM_ORCJAVA-392/src/org/openremote/controller/protocol/socket/UDPListenerCommand.java

     @Override
      public void run() {
         try {
            DatagramSocket dsocket = new DatagramSocket(port);
            byte[] buffer = new byte[2048];
            DatagramPacket packet = new DatagramPacket(buffer, buffer.length);

            // Now loop forever, waiting to receive packets and evaluating them
            while (true) {
              dsocket.receive(packet);
              String msg = new String(buffer, 0, packet.getLength());
              logger.debug("Received UDP packet: " + msg);
              for (Entry<String, Sensor> entry : regexSensorMap.entrySet()) {
                 String regex = entry.getKey();
                 Sensor sensor = entry.getValue();
                 Pattern regexPattern = Pattern.compile(regex);
                 Matcher matcher = regexPattern.matcher(msg);

                // This is a patch from ORCJAVA-392:
                //
                // If a regular expression group is defined and there's a match,
                // the (first) group is returned as the event value instead of just a simple
                // timestamp.

                if (matcher.groupCount() > 0)
                {
                  sensor.update(matcher.group(1));
                }

                else
                {
                  sensor.update("" + System.currentTimeMillis());
                }
              }

              // Reset the length of the packet before reusing it.
              packet.setLength(buffer.length);
            }
          } catch (Exception e) {
            logger.error("Error in UDPListenerThread", e);
          }
      }

I would have to build tests to confirm the correct behavior and it will take me some time before I have a chance to do that.

Let me know...

Posted by juha at Nov 06, 2013 01:55

So, I successfully got this built - eventually. Maybe I'm just not looking in the right place, but I didn't find any current instructions on building from source. Things like svn package names being wrong in the examples, the tools directory being a separate package, and not knowing which tree to use as a base kept me stumbling. In the end, I used package Controller_2_1_0_FM_SNAPSHOT-20130617, and even here I was tripped up by there being no build target for "war". Luckily "run-test" inherently builds the target.

Then, the deployment instructions left me scratching my head when directory names weren't lining up, there being no controller.war file, etc. I ended up just finding the .class files I built and copying them into the binary directory I had previously been running from.

In the end, I was successful in getting this to run, with one minor tweak. I changed this:

                   if (matcher.groupCount() > 0) {
                     sensor.update(matcher.group(1));
                   } else {
                     sensor.update("" + System.currentTimeMillis());
                   }
}

to this:

                 if (matcher.find()) {
                   if (matcher.groupCount() > 0) {
                     sensor.update(matcher.group(1));
                   } else {
                     sensor.update("" + System.currentTimeMillis());
                   }
                 }
}

I'm not a java guy, but it didn't seem like we wanted to grab the matcher info (or assume a non-group match) unless the match was actually found to begin with. Again, maybe I'm missing something.

Oh, and then I had to change my regex (and the data I was sending) when the switch didn't like being set with 1 or 0 - apparently it wants "on" and "off" (and not uppercase, as I found). Maybe I'm missing some more recent documentation somewhere?

In any case, when might this make it into an "official" build of some sort?

Thanks again for your help Juha.

Jeff

Posted by kwazy at Nov 06, 2013 19:10

Hi Jeff,

In any case, when might this make it into an "official" build of some sort?

Yes will do (once I find some free time).

Thanks for doing the work!

– Juha

Posted by juha at Nov 19, 2013 20:31

Updated binary available for testing is here: OpenRemote-Controller-2.1.0_FM_ORCJAVA-392-2.zip

Posted by juha at Feb 27, 2014 16:11
Document generated by Confluence on Jun 05, 2016 09:41