Improved J661 support

Background

Establish the interface requirements for better J661 support, so that J661 and FlightGear can collaborate in a more efficient, and overall better, fashion. So that J661 may connect to a running FlightGear instance in order to use simulated flight data to drive avionics simulated by j661.

Requirements

• use the existing telnet/props interface
• do not touch existing behavior
• make the protocol more efficient to get/set properties
• i.e. avoid polling
• provide some form of push/pull support
• use hashing

Ideas

• • Handling multiple properties **
• One thing that would be great would maybe to be able to have more than one parameter to the output for the client in one telnet output (for example all the properties that have been requested), but even without that, performance should be far better than with polling I think.
• For your information, in the ARINC 661 protocol, the User Application concatenates several "commands" in the same buffer, denoting each specific command by its ID (and in the case of ARINC, also it's widget ID, but it's not relevant to our problem). In our User Application implementation (which happens to be coded in C++), a lot of parameters which are considered as cyclic are send regardless of their value change, because they change often. However the Server checks for their change, and in this case, it does not go further than their decoding. It turned out that the time the Server takes to decode a huge buffer and make its checks is negligible (of the order of 1 ms for example).
• However I need to check the number of parameters which should be sent to the Client in real scenarios. I don't think it's a lot, because ther is further logic in the Client itself, and it could be possible for the Client to unsubscribe for example when a display format is not presented anymore.
• I was also thinking of supporting several parameters (and even return values) at once, my suggestion would be to still consider hex-encoding the UINT hash and then pre-pend it to each property value, the values could then be separated using a normal comma, so that we end up with a list of CSVs.
• In addition, it might be worthwhile to consider pre-pending a UNIX timestamp to each server-side response, so that clients get some timing information, I guess?
• The list of CSVs sounds good to me. And I agree that having the timestamps could be useful in some usecases.
• supporting multiple values in one response (i.e. like CSV) will inevitably require server-side escaping and client-side "unescaping" of properties.

Hashing

• hashing is already supported by the property tree itself (see SimGear library), it is in fact used internally for storing properties, so one of the easiest ways to hashing support would be directly exposing and using the hashes computed by the property tree: http://simgear.sourceforge.net/doxygen/classSGPropertyNode.html It seems, the hash code is stored as an unsigned int (private storage), see line #01758 in http://simgear.sourceforge.net/doxygen/props_8hxx_source.html So it would just be a matter of adding the following to the public interface of the "bucket" class: inline unsigned int getHashcode() const {return hashcode;} And then add a corresponding wrapper to the top level SGPropertyNode class for accessing a node's hash. Either as a static method or as a member function using the "this" pointer. Something along these lines would then allow using hashes of properties internally, which would also reduce bandwidth requirements.

Patch prototype

It turned out to be quite simple actually (as we anticipated already), so we have a working prototype ready. In its current form, it is still rather basic (i.e. no support for multiple properties or hashing), but subscribing and unsubscribing properties does work already. I prepared hashing support but didn't actually implement it fully. So it could definitely be enhanced rather easily if you think that'd be useful. I guess, it's now a matter of coming up with a list of changes that would be useful for j661, so that we can adapt the patch accordingly and submit it to the FG repository for review/inclusion.

    diff --git a/src/Network/props.cxx b/src/Network/props.cxx
    index 7240d5c..bd014db 100644
    --- a/src/Network/props.cxx
    +++ b/src/Network/props.cxx
    @@ -33,6 +33,7 @@
    #include <simgear/misc/strutils.hxx>
    #include <simgear/props/props.hxx>
    #include <simgear/props/props_io.hxx>
    +#include <simgear/structure/exception.hxx>

    #include <sstream>
    #include <iostream>
    @@ -45,6 +46,12 @@

    #include "props.hxx"

    +#include <map>
    +#include <vector>
    +#include <string>
    +
    +#include <boost/functional/hash.hpp> // for property hashing (should probably be replaced eventually)
    +
    using std::stringstream;
    using std::ends;

    @@ -55,7 +62,7 @@ using std::endl;
      * Props connection class.
      * This class represents a connection to props client.
      */
    -class PropsChannel : public simgear::NetChat
    +class PropsChannel : public simgear::NetChat, public SGPropertyChangeListener
    {
         simgear::NetBuffer buffer;

    @@ -75,6 +82,7 @@ public:
          * Constructor.
          */
         PropsChannel();
    +    ~PropsChannel();

         /**
          * Append incoming data to our request buffer.
    @@ -89,10 +97,38 @@ public:
          */
         void foundTerminator();

    +    // callback for registered listeners (subscriptions)
    +    void valueChanged(SGPropertyNode *node);
    private:
    +
    +    typedef std::vector<std::string> ParameterList;
    +
    +
         inline void node_not_found_error( const string& s ) const {
             throw "node '" + s + "' not found";
         }
    +
    +    void error(std::string msg) {  // wrapper: prints errors to STDERR and to the telnet client
    +       push( msg.c_str() ); push( getTerminator() );
    +       SG_LOG(SG_GENERAL, SG_ALERT, __FILE__<<"@" << __LINE__ <<" in " << __FUNCTION__ <<":"<< msg.c_str() << std::endl);
    +    }
    +
    +
    +    bool check_args(ParameterList tok, unsigned int num, const char* func) {
    +       if (tok.size()-1 < num) {
    +          error(string("Wrong argument count for:")+string(func) );
    +          return false;
    +       }
    +       return true;
    +    }
    +
    +    std::vector<SGPropertyNode_ptr> _listeners;
    +    typedef void (PropsChannel::*TelnetCallback) (const ParameterList&);
    +    std::map<std::string, TelnetCallback> callback_map;
    +
    +    // callback implementations:
    +    void subscribe(const ParameterList &p);
    +    void unsubscribe(const ParameterList &p);
    };

    /**
    @@ -104,6 +140,67 @@ PropsChannel::PropsChannel()
           mode(PROMPT)
    {
         setTerminator( "\r\n" );
    +    callback_map["subscribe"]    =    &PropsChannel::subscribe;
    +    callback_map["unsubscribe"]   =   &PropsChannel::unsubscribe;
    +}
    +
    +PropsChannel::~PropsChannel() {
    +  // clean up all registered listeners   
    +  for (unsigned int i=0;i< _listeners.size(); i++)
    +    _listeners[i]->removeChangeListener( this  );
    +
    +}
    +
    +void PropsChannel::subscribe(const ParameterList& param) {
    +   if (! check_args(param,1,"subscribe")) return;
    +
    +   std::string command = param[0];
    +   const char* p = param[1].c_str();
    +   if (!p) return;
    +
    +        SG_LOG(SG_GENERAL, SG_ALERT, p << std::endl);
    +        push( command.c_str() ); push ( " " );
    +        push( p );
    +        push( getTerminator() );
    +
    +        SGPropertyNode *n = globals->get_props()->getNode( p,true );
    +   if ( n->isTied() ) {
    +      error("Error:Tied properties cannot register listeners");
    +      return;
    +   }
    +    if (n) {
    +         n->addChangeListener( this );
    +    std::stringstream hashcode;
    +    boost::hash<std::string> string_hash;
    +    hashcode << "id:" << string_hash(n->getPath()) << getTerminator();
    +    // push( hashcode.str().c_str() );  // not yet very useful
    +    _listeners.push_back( n ); // housekeeping, save for deletion in dtor later on
    +         }
    +         else {
    +       error("listener could not be added");
    +         }
    +
    +
    +}
    +
    +void PropsChannel::unsubscribe(const ParameterList &param) {
    +  if (!check_args(param,1,"unsubscribe")) return;
    +
    +  try {
    +   SGPropertyNode *n = globals->get_props()->getNode( param[1].c_str() );
    +   n->removeChangeListener( this );
    +  } catch (sg_exception &e) {
    +     error("Error:Listener could not be removed");
    +  }
    +}
    +
    +
    +//TODO: provide support for different types of subscriptions MODES ? (child added/removed, thesholds, min/max)
    +void PropsChannel::valueChanged(SGPropertyNode* ptr) {
    +  SG_LOG(SG_GENERAL, SG_ALERT, __FILE__<< "@"<<__LINE__ << ":" << __FUNCTION__ << std::endl); 
    +  std::stringstream response;
    +  response << ptr->getPath(true) << "=" <<  ptr->getStringValue() << "\r\n"; //TODO: use hashes, echo several properties at once, use proper terminator
    +  push( response.str().c_str() );
    }

    /**
    @@ -160,7 +257,7 @@ PropsChannel::foundTerminator()
         const char* cmd = buffer.getData();
         SG_LOG( SG_IO, SG_INFO, "processing command = \"" << cmd << "\"" );

    -    vector<string> tokens = simgear::strutils::split( cmd );
    +    ParameterList tokens = simgear::strutils::split( cmd );

         SGPropertyNode* node = globals->get_props()->getNode( path.c_str() );

    @@ -292,7 +389,7 @@ PropsChannel::foundTerminator()
                         if ( !globals->get_commands()
                                  ->execute( "reinit", &args) )
                         {
    -                        SG_LOG( SG_NETWORK, SG_ALERT,
    +                        SG_LOG( SG_GENERAL, SG_ALERT,
                                     "Command " << tokens[1] << " failed.");
                             if ( mode == PROMPT ) {
                                 tmp += "*failed*";
    @@ -357,7 +454,7 @@ PropsChannel::foundTerminator()
                         if ( !globals->get_commands()
                                  ->execute(tokens[1].c_str(), &args) )
                         {
    -                        SG_LOG( SG_NETWORK, SG_ALERT,
    +                        SG_LOG( SG_GENERAL, SG_ALERT,
                                     "Command " << tokens[1] << " failed.");
                             if ( mode == PROMPT ) {
                                 tmp += "*failed*";
    @@ -386,7 +483,14 @@ PropsChannel::foundTerminator()
                     mode = DATA;
                 } else if ( command == "prompt" ) {
                     mode = PROMPT;
    -            } else {
    +            } else if (command  == "subscribe" || command == "unsubscribe") {
    +         TelnetCallback t = callback_map[ command.c_str() ]; //FIXME: use map lookup for condition
    +         if (t)
    +                 (this->*t) (tokens);
    +         else
    +            error("No matching callback found for command:"+command);
    +       }
    +       else {
                     const char* msg = "\
    Valid commands are:\r\n\
    \r\n\
    @@ -400,7 +504,9 @@ prompt             switch to interactive mode (default)\r\n\
    pwd                display your current path\r\n\
    quit               terminate connection\r\n\
    run <command>      run built in command\r\n\
    -set <var> <val>    set <var> to a new <val>\r\n";
    +set <var> <val>    set <var> to a new <val>\r\n\
    +subscribe <var>      subscribe to property changes (returns hash code for property)\r\n\
    +unscubscribe <id>  unscubscribe from property changes (id must be the property hash as returned from subscribe command)\r\n";
                     push( msg );
                 }
             }
    diff --git a/src/Network/props.hxx b/src/Network/props.hxx
    index e55f7c1..09b5a09 100644
    --- a/src/Network/props.hxx
    +++ b/src/Network/props.hxx
    @@ -40,7 +40,8 @@
      * FlightGear properties.
      */
    class FGProps : public FGProtocol,
    -      public simgear::NetChannel
    +      public simgear::NetChannel,
    +      public SGPropertyChangeListener // for subscriptions
    {
    private:

Related Discussions

• Roadmap for Flightgear connection
• 0.51 beta 2 : FlightGear connection