Creating Complex RPN Expressions for Cacti/rrdtool

Rrdtool is a popular graphing tool for a growing number of platforms that allows time-series statistics to be placed in a database and plotted. Combined with a tool such as Cacti, a number of statistics can be automatically pulled from script, Simple Network Management Protocol (SNMP), or a web-based enterprise management framework (WBEM) such as Windows Management Instrumentation (WMI) or OpenWBEM.  The values pulled from these sources can be stored in rrdtool's database management and used for different service management functions including performance management, incident management, problem management, capacity management, and overall SLA compliance measurement (note that these are terms that are specially defined in the ITIL framework for service management). Cacti/rrdtool also lend themselves well to Six Sigma type reporting for IT services.

From time to time, data in rrdtool needs to be manipulated before it is plotted to improve the quality or accuracy of the information being displayed. This can be due to many factors, from needing to address a scaling issue to transforming data using an arbitrary function. SNMP has an inherent limitation that floating point values cannot be returned (vendors will often overcome this by returning data in the desired level of accuracy as a whole number, ex. 50.00 degrees returned as 5000 hundredths of degrees).  Manipulating data in rrdtool is achieved using a VDEF or CDEF. VDEF functions return a single value/time from a transformation/aggregation of one or more  data series and a CDEF generates a new data series based on the same transformation being applied to every point. Both use RPN syntax to perform the transformation.

Mathematical functions are conventionally written in infix notation (ex. 1+1), but some applications including dc (desk calculator, an extremely old reverse polish notation calculator on the UNIX/Linux platform) and rrdtool require a syntax called Reverse Polish Notation (RPN). RPN is written in postfix notation (i.e. 1,1,+ in rrdtool and 1 1 + in dc). 

There are a couple of formal methods that computer algorithms can use to convert infix notation to postfix notation, but most system administrators dealing with CDEFs and VDEFs may not have access to a tool using one of these formalized methods and may need to work out the formula manually using educated guessing and trial and error. From building a few of these functions, I propose the following guidelines to help build some larger expressions.

Figure out what you are trying to calculate in infix notation.

A lot of people ignore this, both in system administration and elsewhere. If you don't know the equation and you can't calculate it by hand, then there is no point writing a CDEF/VDEF. This will only make things worse. Determine the variables and identify the relationship that needs to be plotted.  

Break the equation into smaller pieces, calculate each piece independently, and combine pieces at the end to form a larger expression.

It is best to convert infix to postfix in smaller pieces. Due to the nature of RPN, direct substitution is allowed without using parenthesis. This is not necessarily the case with infix notation.

Example: Dew point approximation. I recently worked this out for a colleague based on plots of temperature and humidity reported by our cabinet power distribution units in one of our datacenters.

D = 237.7 * ((17.271 * T)/( 237.7 + T) + ln(RH/100)) / ( 17.271 -(( 17.271* T)/(237.7 +T) + ln(RH/100)))

Define gamma as: (( 17.271* T)/(237.7 +T) + ln(RH/100))

T is current tempurature (C) and RH is the current relative humidity as a value ( 0 <= RH <= 100 )

Assume that our rrdtool graph defines 2 data sources, t and r, for temperature (C) and relative humidity respectively. Let's start with the ln(RH/100):

First, divide RH by 100,

r,100,/

Now rrdtool uses a base 10 logarithm for the LOG function, we need a natural logarithm. Use the relation log_b(a) = log(a)/log(b). This is likely the hardest part of the whole conversion.

Note e^1 = 1,EXP

LN(RH/100) = LOG(RH/100)/LOG(e) = r,100,/,LOG,1,EXP,LOG,/

Now, let's complete the expression for gamma.

(( 17.271* T)/(237.7 +T) + ln(RH/100))

Everything in red still needs to be determined. Start with finding the numerator and the denominator separately,

Denominator: 237.7,t,+
Numerator: 17.271,t,*

Now substitute into the RPN expression for division: N,D,/

17.271,t,*,237.7,t,+,/

Now finalize the equation for gamma by adding the logarithm piece that we determined before:

gamma = 17.271,t,*,237.7,t,+,/,r,100,/,LOG,1,EXP,LOG,/

Now we have something in the form of 237.7*gamma/(17.271-gamma). Now work through the numerator and denominator

Numerator: 237.7,17.271,t,*,237.7,t,+,/,r,100,/,LOG,1,EXP,LOG,/,*
Denominator: 17.271,17.271,t,*,237.7,t,+,/,r,100,/,LOG,1,EXP,LOG,/,-

Now build the full function:

237.7,17.271,t,*,237.7,t,+,/,r,100,/,LOG,1,EXP,LOG,/,*,17.271,17.271,t,*,237.7,t,+,/,r,100,/,LOG,1,EXP,LOG,/,-,/

and this yields a general form for the approximation for dew point given temperature and relative humidity using RPN.

Use another tool, such as OpenOffice Calc or Microsoft Excel to calculate check values for the VDEF/CDEF.

This is basic common sense, there is no point in plotting a value if it is not correct. Another calculating tool can help verify pieces of the calculation and validate that RPN is being applied correctly.

Know your SI prefixes and their conversions

Rrdtool uses common SI prefixes ranging from 10^-24 to 10^24. See Wikipedia for a list of these. Conversion up (ex. M -> G) involves division by 1000. Conversion down (ex. M -> k) involves multiplication by 1000.