This page last changed on Feb 02, 2014 by istian.

Hi,

I am struggling with creating a graph.
The issue is that when entering the URL to test my graph the page just comes up blank.

http://xxxxxxx:18581/controller/graph?name=graph2&start=20140202-14-00

All I see is a blank web page....

Also trying to show the graph in iPhone App it just comes up blank.

When starting the controller the test.rrd file is created and there are no errors in the log.
The sensor I would like to graph is a temperature sensor, custom type 9.001 DPI.
This is the XML file:

<rrdDB fileName="test.rrd" step="60">
<datasource name="Utetemperatur" type="GAUGE" heartbeat="120" />
<archive function="AVERAGE" xff="0.5" steps="1" rows="300" />
<archive function="AVERAGE" xff="0.5" steps="10" rows="144" />
<archive function="AVERAGE" xff="0.5" steps="15" rows="672" />
<archive function="AVERAGE" xff="0.5" steps="60" rows="744" />
<archive function="AVERAGE" xff="0.5" steps="120" rows="4380" />
</rrdDB>

<rrd_graph_def name="graph2">
<options>
<vertical_label>Temp (°c)</vertical_label>
<title>TEST</title>
</options>
<datasources>
<def>
<name>Temp_SDB</name>
<rrd>test.rrd</rrd>
<source>Utetemperatur</source>
<cf>AVERAGE</cf>
</def>
</datasources>
<graph>
<line>
<datasource>Temp_SDB</datasource>
<color>#FF0000</color>
<legend>Temp Salle de Bain\r</legend>
<width>2</width>
</line>
<comment>This chart displays OR sensor values.</comment>
</graph>
</rrd_graph_def>

</rrd4j>

I have also tried to open the test.rrd file, and i suspect this is the cause of my problems....
It seems like the logging of data is not working correct???
This is the content of the file (opened in textedit).

RRD4J, version 0.1 <RÓ\PUtetemperatur GAUGE x¯¯@@KAVERAGE ?‡,¯
@@@@@@@@@@¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯AVERAGE ?‡
ê (6Ä)¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯AVERAGE ?‡†@Fį¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯AVERAGE ?‡<Ë@FÄ-¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯AVERAGE ?‡x@FÄ-¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯


screenshot RRD4J.png (image/png)

I am struggling with the same issue, and I found there are many others.
I am able to trace sensors with short polling intervals, which frequently change. I fail with slowly changing signals
like room temperatures.
I think it has to do with data collection interval, and the steps defined in the RRD data source definition. But I don't know yet how and why.

I know other people have been more successful with EmonCMS. There is a tutorial for it on OpenRemote.

PS: can you please remove the database dump from your post; it does not help understanding your problem.

Posted by pz1 at Feb 02, 2014 15:23

Hi Pieter,

I have been looking at the EmonCMS, but I am running the controller on a Synology NAS and I don't know how to
install the EmonCMS Server on the Synology. I know I don't have to run the server my self, so I am considering that as well.

However, does your URL come up as blank as well, or does it at least show labels and x/y axis?
I just get a white web page with nothing in it....

How does your test.rrd file look like? Mine looks funny, as shown above... I support your theory about collection issues.

Posted by istian at Feb 02, 2014 15:39

click on the word struggling in my post to see my setup

Posted by pz1 at Feb 02, 2014 16:23

After I found out that I made a typo in the XML configuration file, which gave me a FileNotFoundException all the time, I am confronted with a similar issue as mentioned above: no graph appears on my Android panel (Note: I am using Android console 2.1.0 SNAPSHOT-2013-03-02), although I don't get any error messages anymore.
I tried to display my graph using the Webview Widget and also the Image Widget. I created a custom sensor linked to the getRrdGraphURL command (as described in the How To about RRD4J). I also tried entering the URL directly in the URL field of the Webview Widget.
Then I placed a Label Widget on my screen to see the URL. The URL displayed in the Label Widget seems to be correct ("http://192.168.3.101:8080/controller/graph?name=graphActTemp&start=20140321-22-15&end=20140322-09-00&width=600&height=326").
When I enter this URL directly in the browser of any of my Windows PC's and laptops, I get a nicely displayed graph in my browser window (see screenhot in the attachment).
However, when I enter the same URL in the browser on my Android tablet, I end up with a browser screen completely filled with characters, starting with PNG IHDR and so on...So it seems that my Android tablet (Android Ice Cream Sandwich version 4.0.4 on a Samsung Galaxy Tab 2 GT-P5110) has a problem with displaying the PNG image of the graph it receives (I found in the source code that the graph created by RRD4J is indeed returned as a PNG image).

Note 1: I'm also using Openremote with Emoncms, which works fine on both my Windows computers and Android tablet. However, Emoncms doesn't use a fixed sized circular database like RRD and for now it doesn't offer the average user many ways to create a personal layout for his graphs (users who have the programming knowledge can modify or add to the source code to suit their needs). Nevertheless, I like Emoncms. The default graphs look already nice and emoncms is pretty straightforward to use.

Note 2: In order to get a nice graph from RRDJ4, make sure you have set the step interval and heartbeat interval in the XML configuration file correctly, depending on the interval that values arrive in your OR sensors, otherwise you'll get a lot of so called UNKNOWN Primary Data Points (PDP's) and consequently UNKNOWN Consolidated Data Points (CDP's) in the archives of your RRD database. These will not be displayed in a graph. See the documentation on internet on RRDtool for this.
(I personally found it a bit of a struggle to get through this documentation to find all the relevant information I needed. Therefore I'm currently working on a small tutorial in wich I try, from what I learned in the documentation on RRDtool, to explain a bit more how RRD processes the samples (values) it receives and the meaning of the parameters that need to be set in the XML configuration file. Of course when it is finished, I would like to share this tutorial with the OR community. It's only a pity that I cannot really use RRD4J on my Android panel at the moment.

Tom

Posted by tom.christiaens at Mar 22, 2014 13:16

While browsing on the internet I came across jira.openremote.org. There I found an issue ORCJAVA-423 (RRD4JServlet doesn't set the content-type heading to 'image/png'), that is marked as resolved and fixed since 1 Feb 2014. As far as I can see, I believe this will be the fix for the problem I have at the moment with displaying an RRD4J image on my Android panel. On JIRA I can see that it is the file RrdGraphServlet.class that needs to be modified, just by adding one simple extra line.
But the problem is that I don't know how to do this. If there were a RrdGraphServlet.class file with the fix, would it then be possible just to replace the existing RrdGraphServlet.class file with the one containing the fix?
If yes, could someone pass me this .class file or explain to me how I can make this file myself?
You must know I am not a (Java) programmer. I easily found out how to decompile a .class file into a .java file (there are a lot of Java decompilers on the internet available), but then, after making modifications or adding code, you need to compile it again, and at this point I get stuck. I tried using javac, but then I get a lot of compile errors, referring to unknown variables etc. So there is obviously more to it than just compile the single .java file.
Perhaps I should just be patient enough and wait until a new release of the OR Controller software is coming out that contains the fix.

Tom.

Posted by tom.christiaens at Mar 23, 2014 17:42

Using the ant build script is easiest to build the controller from its source files.
You could checkout a full controller distribution from subversion, apply the patch (patch -p 1 < patchfile) and just type ant to build the controller.

Posted by ebariaux at Mar 24, 2014 10:27

Eric,
Thanks for the advice. You've put me back on track with RRD4J. It took me some time to understand what you meant by a "checkout from subversion" and how to do that (I've joined Openremote just about five weeks ago and I don't have any real software development experience), but now I have a nice RRD4J PNG-image on my Android panel. Of course also thanks to Rich Turner who made the patch to the RrdGraphServlet class.

In order to do the "checkout" and make the patch I did the following: (after reading in the developer tutorial on www.openremote.org and also googling quite a long time):
1. I installed Subversion 1.8.8 on my Windows computer; see subversion.apache.org (see under binary packages).
2. I installed TortoiseSVN; see tortoisesvn.net.
3. I installed Apache Ant 1.9.3; see ant.apache.org (don't forget to define the environment variable ANT_HOME and update the system variable "path", see the installation instructions on the website).
4. Using TortoiseSVN I could pull the source code from the source code repository and put in a map(i.e. "checkout"). I finally selected the controller 2.1.0 snapshot 2013-06-17 under the "snaphot" map in the repository (the controller 2.1.0 under the "project" map seems not to contain the source code for RRD4J?)
5. I added the missing code line in RrdGraphServlet.java (see JIRA, ORCJAVA-423) and saved the file.
6. In the command shell (cmd.exe in Windows) I placed the command prompt in the directory containing the source code and the build.xml file. Then I entered "ant" at the command prompt, which gave me a successfull build.
7. First I tried to install the complete build, but that gave me a couple of errors and the controller didn't want to synchronize with the Online Designer. So then I just replaced the old RrdGraphServlet.class file with the newly built one in my existing controller installation. Then I could restart my controller without any error (besides the ever repeating error that the Vera Client couldn't be started, but that doesn't seem to have any consequence for me). And now I can see the RRD4J image through the Webview widget on my Android panel. Fantastic!

Tom.

Posted by tom.christiaens at Mar 24, 2014 19:42

Glad you have it working and thanks for sharing your detailed steps on how to achieve it.

Posted by ebariaux at Mar 25, 2014 15:08

Since the release of the Pro version 1.0.13 controller, the graphs do appear on my Android console using the RRDGraph as sensor for the Webelement.
I guess this is due to this fix?

Fixed: RRD4JServlet doesn't set the content-type heading to 'image/png' (ORCJAVA-423)
Posted by pz1 at May 13, 2014 13:24

Yes, most browser did seem to display the image with the content type not set correctly, but not Android, which did not display anything.
With the content type now properly set, the image is displayed.

Posted by ebariaux at May 14, 2014 13:41

File test.rrd seems incompatible with rrdtool:
To get some more insight in the recorded data I did install the rrdtool version 1.2.30 on my Synology DS212+. (installed it using IKPG).
Using the tool I tried to make a XML dump of the data. Unfortunately the tool says that the OpenRemote rrd4j test.rrd is not a proper RRD file.
Anyone who has a clue what might be wrong?
I am interested in RRD to see if it can be used as some kind of a signal processor for data that I feed in my rules.
I noticed that some rules are affected by spurious readings on certain sensors. The RRD Average function might help there.

Posted by pz1 at May 14, 2014 19:42

The OpenRemote rrd database is created with the java lib rrd4j: https://code.google.com/p/rrd4j/
I am not sure if the version is compatible.

Posted by mredeker at May 15, 2014 08:58

Thanks Marcus.
Might indeed be a version issue. Your link points to rrd4j version 2.2.
IPKG is not always up to date. With IPKG I hoped to avoid going through the agony of compiling it myself
Pieter

Posted by pz1 at May 15, 2014 09:08

The version used in the Pro controller is 2.1.1.

Posted by mredeker at May 15, 2014 09:40

Tried to dump to an XML file with a more recent RRDTool version on Raspberry Pi:

pi@raspberry ~ $ /usr/bin/rrdtool version
RRDtool 1.4.7  Copyright 1997-2012 by Tobias Oetiker <tobi@oetiker.ch>
               Compiled Sep  4 2012 23:43:06

Usage: rrdtool [options] command command_options
Valid commands: create, update, updatev, graph, graphv,  dump, restore,
                last, lastupdate, first, info, fetch, tune,
                resize, xport, flushcached

pi@raspberry ~ $ sudo /usr/bin/rrdtool dump test.rrd
ERROR: 'test.rrd' is not an RRD file
pi@raspberry ~ $

There is only one later version 1.4.8 from May 2013. Using windows7 explorer I did copy test.rrd from my DS212+ to the Pi.

Posted by pz1 at May 15, 2014 10:45

Could be that the java version is not completely compatible with the C++ version

Posted by mredeker at May 15, 2014 10:58

Could quite well be the case (http://en.wikipedia.org/wiki/Rrd4j).
That makes it very difficult to get data out of the RRD database. I give up on this tool
Given the dormant status of OpenRemote I don't expect any RRD extensions to come soon.
Thanks anyhow for your responses.

Pieter

edit: changed link to a better reference

Posted by pz1 at May 15, 2014 11:12
Document generated by Confluence on Jun 05, 2016 09:36