FlightGear wiki:Instant-Refs: Difference between revisions

From FlightGear wiki
Jump to navigation Jump to search
m (Add localized name to stop GreasyFork from complaining)
Line 351: Line 351:
<syntaxhighlight lang="javascript" enclose="div">// ==UserScript==
<syntaxhighlight lang="javascript" enclose="div">// ==UserScript==
// @name        Instant-Cquotes
// @name        Instant-Cquotes
// @name:it    Instant-Cquotes
// @version    0.34
// @version    0.34
// @description Automatically converts selected FlightGear mailing list and forum quotes into post-processed MediaWiki markup (i.e. cquotes).
// @description Automatically converts selected FlightGear mailing list and forum quotes into post-processed MediaWiki markup (i.e. cquotes).

Revision as of 18:51, 6 May 2016

Quotes-logo-200x200.png
Instant-Cquotes script in Firefox
Instant-Cquotes screenshot prototyping runtime format selection

The Instant-Cquotes script is a browser addon (user script) implemented in JavaScript in order to convert excerpts (created via copy&paste) from FlightGear forum or mailing list postings into MediaWiki markup/quotes to be used on the FlightGear wiki. It is supported by Firefox, Google Chrome/Chromium, Opera and Safari. It is being developed and maintained by a group of volunteers involved in maintaining the wiki and in trying to provide more up-to-date information to end-users who may not be as involved in the various FlightGear-related communication channels.

Background and motivation

FlightGear's development is, at best, "self-coordinated", meaning that contributors discuss ideas and make proposals to contribute in a certain fashion and then team up to implement certain features and building blocks, often just temporarily.

Unfortunately, due to a lack of development manpower, many ideas are not implemented immediately; it is, thus, important to know their pros and cons, as well as who originally proposed them and/or might help with their implementation, even after a long time, preferably with links to the original discussions so that new contributors can decide whether to get involved in some effort or not. The project documentation, however, is in great need of improvement[1] and usually significantly lacking behind:[2] many core developers update it seldomly, if not anymore, as it takes time to write it, as well as an understanding of the inner workings of a complex system such as FlightGear. Furthermore, this task might not be attractive due to its short term impact on the project,[3] and it is overwhelming to think about creating documentation that would address the needs of many different kinds of contributors with different backgrounds, experience levels and goals.[4]

Forum and mailing lists discussions have therefore become the only up-to-date (albeit difficult to filter)[5] source of information about recent development progress; this makes it tricky to know what is going on, what needs fixing, what were the decisions taken by the developers.[6]

The aim of the Instant-Cquotes script is to help wiki editors to copy relevant excerpts from such sources, formatting them as proper quotations, and bootstrap new articles collecting them until a dedicated rewrite is made. It can also be used to reuse announcements to update the changelogs, newsletter or the Release plan/Lessons learned page.

In a few cases, such collections of quotes helped not only create bootstrap new articles, but even actual features.

In other cases, quotes have been used to update documentation of features (e.g. Rembrandt) whose maintainers may not be actively involved in FlightGear, to help document discussions that are taking place in the meantime, and provide some background information for people interested in the corresponding feature.

Installation

  1. Install a user script manager. On Firefox, you can use Greasemonkey; on Chrome/Chromium, Opera or Safari, you can use Tampermonkey (download links: Chrome/Chromium, Opera, Safari).
  2. Visit the Instant-Cquotes page on GreasyFork and click on Install this script (green button). If Greasemonkey/Tampermonkey prompts you to confirm the installation, agree to do so.
click the green button to install the script

Manual installation

Note  This will install the most recent development version of the script, which might contain bugs. Also, GreaseMonkey/TamperMonkey will not update it automatically whenever a new version is released.
  • Firefox
  1. Install Greasemonkey.
  2. Save the script below as instant_cquotes.user.js, then drag-and-drop it into Firefox.
Screenshot showing the Greasemonkey setup dialog (on Firefox)
  • Chrome/Chromium, Opera, or Safari
  1. Install Tampermonkey.
  2. Navigate to Add a new Script.
  3. Copy and paste the script below into the editing window.
  4. Click the Save button (just above the Search button).

Mobile installation

As of May 2016, there is no separate version available for mobile use. Your best chance is installing a userscript addon on Android, like one of those:

For installation instructions, refer to Tampermonkey for Android or How To Access Greasemonkey Scripts on Android Phones.

Testing/feedback would obviously be appreciated - if in doubt, feel free to just edit the wiki page to add your findings/questions.

Configuration

GreaseMonkey menu shown in FireFox with instanct cquotes menu items
The configuration dialog for the Instant-Cquotes script

As of version 0.30, a dedicated configuration dialog is in the process of being added, so that certain script features can be dynamically configured, without having to edit the script. For now, this is merely a placeholder that provides a checkbox to easily enable/disable the debug mode. In the future, we are hoping to also expose other features this way.

Usage

Instant-Cquotes script, with updates contributed by Red Leader
Screenshot showing Instant-Cquotes 0.30 at work
  1. Go to some mailing list archive URL, for example [1] or any forum message, such as Template:Forumref.
  2. Select the relevant portion of text.
  3. When you release the mouse button, a box will appear containing the converted text (for now, mainly properly-referenced quotes for the wiki).
  4. As the text will already be selected for you, press Ctrl+C to copy it (no longer necessary).
  5. Paste the text into the desired wiki page.

For example, by selecting part of the forum post in the link above you can get the following quotation:

Cquote1.png The upcoming FlightGear version (3.2) will contain a canvas-based map dialog, including a modular "plugin" system for creating custom map layers and charts with roughly ~50 lines of code, most of it boilerplate.

This is entirely XML/Nasal based (scripted) - symbols can be pretty much anything, raster or vector images (png or svg), but even animated. Styling can be customied, too. For more info, I suggest to check out: MapStructure#Porting the map dialog

MapStructureDialog.png
— Hooray (Jun 14th, 2014). Re: Get objects to show up on Map/Radar.
(powered by Instant-Cquotes)
 Cquote2.png

On quoting

1rightarrow.png See FlightGear wiki:Quoting Guidelines for the main article about this subject.

Using the Instant-Cquotes script is a good way to bootstrap and write some preliminary notes; however, while quotes might be useful to understand how undocumented subsystems and features work and are definitely better than nothing, they are not meant to replace proper, structured and well-written wiki articles.[7]

One way to convert pages bootstrapped using quotes is to extract relevant information from them and keep citations only as references; in case important details are missing, they can be asked for on the mailing lists (on the forum, the chance to get a complete answer might be lower).[8] Another option might be moving the quotes to the Talk page for each entry, which would preserve the sources without clogging up the articles.[9]

As a matter of fact, the whole paragraph above was assembled using this approach; to see for yourself, look up the references at the end of this page. For another example, see TerraSync#News.

Development

Note  A Chrome/Chromium-specific extension that will not need Tampermonkey installed is under development.

Adding sources

WIP.png Work in progress
This article or section will be worked on in the upcoming hours or days.
See history for the latest developments.

Adding a new source is pretty straightforward if you understand how xpath and regexes work - basically, you only need an archive (e.g. gmane), and then determine the xpath of each relevant field, as well as the regular expression to process the extracted fields (optional).

The basic steps are these:

  1. open the user script in an editor
  2. navigate to the meta header of the user script
  3. add a new URL to the top of the script, e.g. by copying/adapting an existing line like this:
// @match       https://sourceforge.net/p/flightgear/mailman/*

Once copied, add the new URL: 

// @match       http://thread.gmane.org/gmane.games.flightgear.devel/*

Next, you need to navigate to the configuration hash to add a new website to it. Again, it makes sense to simply take an existing configuration hash and adapt it as needed (ignore/omit the tests vector for now by keeping it empty): 


  'Sourceforge Mailing list': {
    type: 'archive',
    event: 'document.onmouseup', // when to invoke the event handler
    event_handler: instantCquote, // the event handler to be invoked
    url_reg: '^(http|https)://sourceforge.net/p/flightgear/mailman/.*/',
    content: {
      selection: getSelectedText,
      idStyle: /msg[0-9]{8}/,
      parentTag: [
        'tagName',
        'PRE'
      ]
    }, // content recipe
    // vector with tests to be executed for sanity checks (unit testing)
    tests: [
    ], // end of vector with self-tests
    // regex/xpath and transformations for extracting various required fields
    author: {
      xpath: 'tbody/tr[1]/td/div/small/text()',
      transform: extract(/From: (.*) <.*@.*>/)
    },
    title: {
      xpath: 'tbody/tr[1]/td/div/div[1]/b/a/text()'
    },
    date: {
      xpath: 'tbody/tr[1]/td/div/small/text()',
      transform: extract(/- (.*-.*-.*) /)
    },
    url: {
      xpath: 'tbody/tr[1]/td/div/div[1]/b/a/@href',
      transform: prepend('https://sourceforge.net')
    }
  }, // end of mailing list profile

Now, we need to review/adapt the profile according to the new archive we'd like to see supported.

for starters, that means:

  • changing the name of the profile, e.g. to read gmane (instead of sourceforge)
  • change the url_reg field to the gmane URL (this can be a regular expression)

Next, it makes sense to use an XPath checker, so that we can look up the xpath expression for various HTML elements, and add those to the configuration hash above.

For testing purposes, you will probably go to the setup dialog and enable the DEBUG mode, and use your browser's console to see what is going on.

Getting involved

While having some experience with JavaScript/HTML and jQuery will definitely be useful, JavaScript is close enough to FlightGear scripting (Nasal), so that people can get involved pretty easily.

Most maintenance work will typically involve reviewing/maintaining a few configuration hashes, that contain meta information for each supported archive (mailing list/forum).

Usually, each hash contains a combination of xpath/regex expressions to look up the relevant information, as well as vector of optional transformations that are applied (in order) to convert contents to a different format (e.g. dates).

In addition, there is growing library of utility functions, a handful wrappers for useful stuff, for example:

  • Host.dbLog(message_string) - log a message to the console if the DEBUG flag is set
  • Host.download(url, callback, method='GET') - will download the URL and pass the downloaded content to the callback specified
  • Host.downloadPosting(url, callback) - will download the posting URL and pass the extracted and transformed author/content and date fields in a hash to the callback specified, the URL must be one supported in the CONFIG hash (i.e. forum/sourceforge for now)
  • Host.make_doc(string, type="text/html") - will turn a string/blob into a DOM that can be queried
  • Host.eval_xpath(document, xpath_expression, type=XPathResult.STRING_TYPE) - will apply the xpath expression to the document specified, returning the requested type (defaulted to string)
  • Host.set_persistent(key,value) - stores a key/value pair
  • Host.get_persistent(key,default_value) - retrieves a values using the key specified, falling back to the default value

Porting

Porting the script to support other browsers/script engines is greatly appreciated. Typically, this should be pretty self-contained, because all main APIs are intended to be encapsulated in a so called "Environment" hash, where APIs that are specific to a particular browser/script engine should be provided with a wrapper. As of mid 2016, most APIs are now kept inside such an Environment hash (look at the GreaseMonkey hash for reference/details), so that it is now even possible to turn the script into a standalone FireFox addon without having to change much of the underlying code.

Self checks (unit testing)

For regression testing purposes, there's a dedicated "self check" dialog that will be extended over time. For now it will download a few profile/website specific postings using a vector called "tests" and then log the posting's title, author and date to the console.

The next step will be actually showing that information in the dialog itself - there's a separate helper function to accomplish that, so that the corresponding div layer can be updated with the results.

automatically executed sanity checks

Debug mode

Instant-Cquotes debug mode


AJAX (live page editing)

For now, the setup dialog will try to obtain a login token for the wiki and show a message if successful. In addition, the profile/website hash also contains a new wiki entry for the FlightGear wiki, whose event_handler callback will be invoked once the FG wiki is visited - the console/log will show a greeting, so that is where other code can be added - e.g. to help clean up/rewrite FGCquote-based articles automatically etc.

See User:Red Leader/Sandbox/AJAX test

Mobile edition

Note  As of 02/2016, Hooray is contemplating to make this available as an addon for Android phones.

Issues/limitations

Bugs

  • It's eating characters, apparently related to regex/xpath handling - e.g. words like "analyzing" are turned into "analying" [2]

Non working URLs

Feature requests & ideas

  • expose the cquote/ref markup via the UI so that it can be edited/customized and treated like a template
  • identify common/repeated links and automatically create link/infrastructure templates and use those (should be straightforward using the AJAX mode) [3]
  • add a devel/maintainer mode where it will return the xpath for a selection [4] [5]
  • move openlink,dblog helpers to Environment hash Done Done
  • identify CLI arguments like --aircraft=c172p and wrap them in between code tags --aircraft=c172p [6] (note that we can simply download options.xml via openlink() and use that, which is kinda of neat...) 60}% completed (see downloadOptionsXML() in the code)
  • introduce "layouts" (templates) for different purposes: newsletter, changelog, wiki article, The Manual (LaTex)  ?
  • use wikipedia template if possible [7]
  • the new tests vector could also contain vectors for tests to test the extract/transform* utilities, see Environment.APITests 50}% completed
  • move environment specific APIs (browser, script host etc) into some kind of Environment hash to encapsulate things (Red Leader was working on a pure Chrome-only version at some point IIRC) 80}% completed
  • encode script settings in created markup, for future processing/updating of quotes
  • look up [x] references and replace with the corresponding link (titled) [8] [9]
    • convert footnotes into Abbr templates [10]
  • support named refs for combining identical refs [11]
  • adopt Template:Forumref
  • implement a less obnoxious quoting mode, without quotes, where only the ref part would be added, e.g. see the example at Graphics Card Profiles (it's still 99% quotes, but much less annoying) Done Done
  • attachment support: identify attachments and link to them: [12] [13] [14] [15]
  • bulletin points: if there is a colon (:) followed by at least two dashes (-), split up everything after the colon to turn each dash into an asterisk (wiki markup for bulletin points), followed by a newline [16]
  • generic URL/template matching, e.g. for for sourceforge commit IDs
  • make filters/conversions configurable via checkboxes (nowiki, wrap in alert/note boxes)
  • make syntax highlighting configurable (language, mode) ?
  • consider using something like the Roles template to turn contributor names into tooltips where contributor roles are shown (core dev, fgdata committer etc)
  • introduce support for tag clouds to help categorize/classify related quotes
  • consider making transformations optional/configurable using check boxes in the jQuery dialog
  • add new input method, for quotes that need to be updated/converted (added script version specifically for this purpose), should also add extraction/processing date
  • investigate why not all mailing list archives/postings are supported correctly: http://sourceforge.net/p/flightgear/mailman/message/8090479/ (problem traced to getPostID())
  • explore having a ref-only mode without using cquotes, i.e. just copy/paste quotes with proper ref tags and a references section, to rewrite the whole thing (possibly with templates for different purposes, e.g. newsletter/changelog) Done Done
  • should use Template:Forumref (category:link templates)
  • should be updated to use the new repo/flightgear file templates created by Johan & RedLeader Not done Not done
  • resolve links to forum threads to look up the title for the topic/posting, so that the posting's title can be used for those links, instead of just the URL - we can probably do that by making an AJAX call to open URLs asynchronously and extract the title for the thread/posting to come up with something like A call to developers-Lockheed -L188 Electra ([17]) 50}% completed
  • convert quoted bug tracker URLs to use the issue template on the wiki Not done Not done
  • do regex/xpath validation, and display any errors (e.g. template/theme changes on the forum would currently break the script) 60}% completed
  • increased focus on supporting different output formats, maybe using a simple jQuery based wizard (wiki, forum, newsletter, changelog) Not done Not done
  • token matching for keywords/acronyms to link to the corresponding wiki articles (e.g. Nasal, Canvas, FG_ROOT, FG_HOME etc) Not done Not done
  • Add support for tickets, merge requests comments and FGUK forum. (also see FlightGear wiki talk:Instant-Cquotes#more sources)
  • GET-encoded SID arguments should be stripped from forum URLs. Not done Not done
  • Links to repositories should be converted to use wiki templates. Not done Not done
  • The regexes used may fail if the HTML DOM of the source changes (e.g., phpBB/theme update)
    • Show a warning when that's the case. 70}% completed (see the self-check dialog)
    • Try multiple regexes in order. 30}% completed (see the self-check dialog)
  • Use the script to update previously created Cquotes automatically
    • Instead of using the getSelection() helper, we could register a match for wiki.flightgear.org with action=edit set, so that we can directly process all text of an edited page, using AJAX calls to open the URL in the background. 40}% completed (see the self-check dialog, available via the greasemonkey menu)
    • See MW:API:Edit § Editing via Ajax

Changelog

Note  Contributors are invited to document their changes here, please also add your wiki handle so that others can more easily get in touch.
  • first stab at implementing unit tests by adding a vector with URLS to be downloaded and fields to be matched (WIP), shown in a jQuery dialog
  • support for persistent settings and a jQuery setup dialog with persistence
  • hosting is moved, to allow auto-updates [18] [19] [20]
  • changed to ref-only quotes for now, not using the FGCquote template anymore, due to its obnoxious appearance on quote-heavy pages (should probably become a runtime option instead)
  • updated to use https for forum postings
  • add version info to each created quote, i.e. for future updates
  • add helper for opening websites asynchronously using AJAX (also via GM helper API)
  • display version number in output dialog
  • begin using the GreaseMonkey API for setting clipboard content

The script

Public domain This work has been released into the public domain by its author, FlightGear contributors. This applies worldwide.
In some countries this may not be legally possible; if so:
FlightGear contributors grants anyone the right to use this work for any purpose, without any conditions, unless such conditions are required by law.
Note  Anybody interested in contributing to the code is invited to directly edit this wiki article. From 05/2016, the script is hosted on GreasyFork to allow automatic updates. If you'd like to see your changes applied, please bump the version number and Elgaton will upload it in the state it was when the version number was bumped. Make sure to perform thorough testing before the bump to prevent unexpected breakage; it is generally a good idea to validate your changes using an online syntax checker, e.g.:

Thank you!

Changes that should be mentioned in the changelog, should be added below (and moved to the #Changelog section subsequently:

  • preparations for adding support to download fgdata related files like options.xml to automatically regex known CLI commands
  • preparatory work for adding a speech-rewrite engine to assist in converting 1st person speech to 3rd person
  • framework for encapsulating userscript specifics in an environment hash to provide better updating/porting support


// ==UserScript==
// @name        Instant-Cquotes
// @name:it     Instant-Cquotes
// @version     0.34
// @description Automatically converts selected FlightGear mailing list and forum quotes into post-processed MediaWiki markup (i.e. cquotes).
// @description:it Converte automaticamente citazioni dalla mailing list e dal forum di FlightGear in marcatori MediaWiki (cquote).
// @author      Hooray, bigstones, Philosopher, Red Leader & Elgaton (2013-2016)
// @icon        http://wiki.flightgear.org/images/2/25/Quotes-logo-200x200.png
// @match       https://sourceforge.net/p/flightgear/mailman/*
// @match       http://sourceforge.net/p/flightgear/mailman/*
// @match       https://forum.flightgear.org/*
// @match       http://wiki.flightgear.org/*
// @namespace   http://wiki.flightgear.org/FlightGear_wiki:Instant-Cquotes
// @run-at      document-end
// @require     https://code.jquery.com/jquery-1.10.2.js
// @require     https://code.jquery.com/ui/1.11.4/jquery-ui.js
// @resource    jQUI_CSS https://code.jquery.com/ui/1.11.4/themes/smoothness/jquery-ui.css
// @resource    myLogo http://wiki.flightgear.org/images/2/25/Quotes-logo-200x200.png
// @grant       GM_registerMenuCommand
// @grant       GM_setValue
// @grant       GM_getValue
// @grant       GM_addStyle
// @grant       GM_getResourceText
// @grant       GM_getResourceURL
// @grant       GM_setClipboard
// @grant       GM_xmlhttpRequest
// @noframes
// ==/UserScript==
//
// This work has been released into the public domain by their authors. This
// applies worldwide.
// In some countries this may not be legally possible; if so:
// The authors grant anyone the right to use this work for any purpose, without
// any conditions, unless such conditions are required by law.
//

'use strict';

// TODO: move to GreaseMonkey host
// prevent conflicts with jQuery used on webpages: https://wiki.greasespot.net/Third-Party_Libraries#jQuery
this.$ = this.jQuery = jQuery.noConflict(true);

// set this to true continue working on the new mode supporting
// asynchronous content fetching via AJAX
// var USE_NG = false;


// This hash is intended to help encapsulate platform specifics (browser/scripting host)
// Ideally, all APIs that are platform specific should be kept here
// This should make it much easier to update/port and maintain the script in the future
var Environment = {
  getHost: function() {
    
    // This will determine the script engine in use: http://stackoverflow.com/questions/27487828/how-to-detect-if-a-userscript-is-installed-from-the-chrome-store
    if (typeof(GM_info) === 'undefined') {
    Environment.scriptEngine = "plain Chrome (Or Opera, or scriptish, or Safari, or rarer)";
    // See http://stackoverflow.com/a/2401861/331508 for optional browser sniffing code.
   }
   else {
    Environment.scriptEngine = GM_info.scriptHandler  ||  "Greasemonkey";
    }
   console.log ('Instant cquotes is running on ' + Environment.scriptEngine + '.');
    
    // See also: https://wiki.greasespot.net/Cross-browser_userscripting
    return Environment.GreaseMonkey; // return the only/default host (for now)
  },
  
  validate: function(host) {
    if(Environment.scriptEngine !== "Greasemonkey" && host.get_persistent('startup.disable_validation',false)!==true) 
      alert("NOTE: This script has not been tested with script engines other than GreaseMonkey recently!");
  },
  
  // this contains unit tests for checking crucial APIs that must work for the script to work correctly
  // for the time being, most of these are stubs waiting to be filled in
  // for a working example, refer to the JSON test at the end
  // TODO: add jQuery tests
  APITests: [
     {name:'download', test: function(recipient) {recipient(true);}  },
     {name:'make_doc', test: function(recipient) { recipient(true);}   },
     {name:'eval_xpath', test: function(recipient) { recipient(true);} },
     {name:'JSON de/serialization', test: function(recipient) {
       //console.log("running json test");
       var identifier = 'unit_tests.json_serialization';
       var hash1 = {x:1,y:2,z:3};
       Host.set_persistent(identifier, hash1, true);
       var hash2 = Host.get_persistent(identifier,null,true);
       
       recipient(JSON.stringify(hash1) === JSON.stringify(hash2));
     } // callback 
     },
    
    // downloads a posting and tries to transform it to 3rd person speech ...
    // TODO: add another test to check forum postings
    {name:'text/speech transformation', test: function(recipient) {
    
    // the posting we want to download
    var url='https://sourceforge.net/p/flightgear/mailman/message/35066974/';
    Host.downloadPosting(url, function (result) {
      
    // only process the first sentence by using comma/dot as delimiter
    var firstSentence = result.posting.substring(result.posting.indexOf(',')+1, result.posting.indexOf('.'));
      
    var transformed = transformSpeech(firstSentence, result.author, null, speechTransformations );
    console.log("3rd person speech transformation:\n"+transformed);   
    
    recipient(true);
    }); // downloadPosting() 
        
  }// test()
    } // end of speech transform test
    
  ], // end of APITests
  
  runAPITests: function(host, recipient) {
    console.log("Running API tests");
    for(let test of Environment.APITests ) {
      //var test = Environment.APITests[t];
      // invoke the callback passed, with the hash containing the test specs, so that the console/log or a div can be updated showing the test results
      //callback(test);
      recipient.call(undefined, test);
      //test.call(undefined, callback)
    } // foreach test
  }, // runAPITests
  
  ///////////////////////////////////////
  // supported  script engines:
  ///////////////////////////////////////
  
  GreaseMonkey: {
  // TODO: move environment specific initialization code here  
  init: function() {
  // Check if Greasemonkey/Tampermonkey is available
  try {
  // TODO: add version check for clipboard API and check for TamperMonkey/Scriptish equivalents ?
  GM_addStyle(GM_getResourceText('jQUI_CSS'));
  } // try
  catch (error) {
  console.log('Could not add style or determine script version');
  } // catch

  var commands = [
  {name:'Setup quotes',callback:setupDialog, hook:'S' },
  {name:'Check quotes',callback:selfCheckDialog, hook:'C' }
  ];
      
  for (let c of commands ) {
   this.registerMenuCommand(c.name, c.callback, c.hook);
  }  
     
  }, // init()
    
  getScriptVersion: function() {
  return GM_info.script.version;  
  },
    
  dbLog: function (message) {
  if (Boolean(DEBUG)) {
    console.log('Instant cquotes:' + message);
  }
  }, // dbLog()
    
  registerMenuCommand: function(name,callback,hook) {
  // https://wiki.greasespot.net/GM_registerMenuCommand
  // https://wiki.greasespot.net/Greasemonkey_Manual:Monkey_Menu#The_Menu
    GM_registerMenuCommand(name, callback, hook);
  }, //registerMenuCommand()

    
   download: function (url, callback, method='GET') {
  // http://wiki.greasespot.net/GM_xmlhttpRequest
     try {
  GM_xmlhttpRequest({
    method: method,
    url: url,
    onload: callback
  });
     }catch(e) {
       console.log("download did not work");
     }
  }, // download()
    
    // is only intended to work with archives supported by the  hash
    downloadPosting: function (url, EventHandler) {
      
    Host.download(url, function (response) {
    var profile = getProfile(url);
    var blob = response.responseText;
    var doc = Host.make_doc(blob,'text/html'); 
    
    var xpath_author = '//'+profile.author.xpath; 
    var author = Host.eval_xpath(doc, xpath_author).stringValue;
    author = profile.author.transform(author); 
      
    var xpath_date = '//' + profile.date.xpath;
    var date = Host.eval_xpath(doc, xpath_date).stringValue;
    date = profile.date.transform(date);
      
    var xpath_posting = '//'+profile.content.xpath;  
    var posting = Host.eval_xpath(doc, xpath_posting).stringValue;

    var result = {author:author, date:date, posting:posting};
      
    EventHandler(result);
      
    }); // AJAX callback
  
      
    }, // downloadPosting()

  
    // turn a string/text blob into a DOM tree that can be queried (e.g. for xpath expressions)
    // FIXME: this is browser specific not GM specific ...
    make_doc: function(text, type='text/html') {
      // to support other browsers, see: https://developer.mozilla.org/en/docs/Web/API/DOMParser
      return new DOMParser().parseFromString(text,type);
    }, // make DOM document
    
    // xpath handling may be handled separately depending on browser/platform, so better encapsulate this
    // FIXME: this is browser specific not GM specific ...
    eval_xpath: function(doc, xpath, type=XPathResult.STRING_TYPE) {
      return doc.evaluate(xpath, doc, null, type, null);
    }, // eval_xpath
    
    set_persistent: function(key, value, json=false) 
    {
      // transparently stringify to json
      if(json) {
        // http://stackoverflow.com/questions/16682150/store-a-persistent-list-between-sessions
        value = JSON.stringify (value);
      }
      
      // https://wiki.greasespot.net/GM_setValue
      GM_setValue(key, value);
    }, // set_persistent
    
    get_persistent: function(key, default_value, json=false) {
     // https://wiki.greasespot.net/GM_getValue
    
      var value=GM_getValue(key, default_value);
      // transparently support JSON: http://stackoverflow.com/questions/16682150/store-a-persistent-list-between-sessions
      if(json) {
        value = JSON.parse (value)  ||  {};
      }
      return value;
    }, // get_persistent

   setClipboard: function(msg) {
   // this being a greasemonkey user-script, we are not 
   // subject to usual browser restrictions
   // http://wiki.greasespot.net/GM_setClipboard
   GM_setClipboard(msg);
  } // setClipboard()

    
  } // end of GreaseMonkey environment, add other environments below
  
}; // Environment hash - intended to help encapsulate host specific stuff (APIs)


// the first thing we need to do is to determine what APIs are available
// and store everything in a Host hash, which is used for API lookups
// the Host hash contains all platform/browser-specific APIs
var Host = Environment.getHost();
Host.init(); // run environment specific initialization code (e.g. logic for GreaseMonkey setup)


// move DEBUG handling to a persistent configuration flag so that we can configure this using a jQuery dialog (defaulted to false)
// TODO: move DEBUG variable to Environment hash / init() routine
var DEBUG = Host.get_persistent('debug_mode_enabled', false);
Host.dbLog("Debug mode is:"+DEBUG);
function DEBUG_mode() {
  // reset script invocation counter for testing purposes
  Host.dbLog('Resetting script invocation counter');
  Host.set_persistent(GM_info.script.version, 0);
}


if (DEBUG)
DEBUG_mode();

// downloadOptionsXML();


// hash with supported websites/URLs,  includes xpath and regex expressions to extract certain fields, and optional transformations for post-processing

var CONFIG = {
  // WIP: the first entry is special, i.e. it's not an actual list archive (source), but only added here so that the same script can be used
  // for editing the FlightGear wiki
  
  'FlightGear wiki': {
    type: 'wiki',
    enabled: false,
    event: 'document.onmouseup', // when to invoke the event handler
    event_handler: function () {
      console.log('FlightGear wiki handler active (waiting to be populated)');
      // this is where the logic for a wiki mode can be added over time (for now, it's a NOP)
      
    var editSections = document.getElementsByClassName('mw-editsection');
    console.log('FlightGear wiki article, number of edit sections: '+editSections.length);
   
    // for now, just rewrite edit sections and add a note to them
   
     [].forEach.call(editSections, function (sec) {
       sec.appendChild(
         document.createTextNode(' (instant-cquotes is lurking) ')
       );
     }); //forEach section
      
    }, // the event handler to be invoked
    url_reg: '^(http|https)://wiki.flightgear.org' // ignore: not currently used by the wiki mode
  }, // end of wiki profile
  
  'Sourceforge Mailing list': {
    enabled: true,
    type: 'archive',
    event: 'document.onmouseup', // when to invoke the event handler
    event_handler: instantCquote, // the event handler to be invoked
    url_reg: '^(http|https)://sourceforge.net/p/flightgear/mailman/.*/',
    content: {
      xpath: 'tbody/tr[2]/td/pre/text()', // NOTE this is only used by the downloadPosting  helper to retrieve the posting without having a selection 
      selection: getSelectedText,
      idStyle: /msg[0-9]{8}/,
      parentTag: [
        'tagName',
        'PRE'
      ]
    }, // content recipe
    // vector with tests to be executed for sanity checks (unit testing)
    tests: [
      {
        url: 'https://sourceforge.net/p/flightgear/mailman/message/35059454/',
        author: 'Erik Hofman',
        title: 'Re: [Flightgear-devel] Auto altimeter setting at startup (?)'
      },
      {
        url: 'https://sourceforge.net/p/flightgear/mailman/message/35059961/',
        author: 'Ludovic Brenta',
        title: 'Re: [Flightgear-devel] dual-control-tools and the limit on packet size'
      },
      {
        url: 'https://sourceforge.net/p/flightgear/mailman/message/20014126/',
        author: 'Tim Moore',
        title: 'Re: [Flightgear-devel] Cockpit displays (rendering, modelling)'
      },
      {
        url: 'https://sourceforge.net/p/flightgear/mailman/message/23518343/',
        author: 'Tim Moore',
        title: '[Flightgear-devel] Atmosphere patch from John Denker'
      } // add other tests below

    ], // end of vector with self-tests
    // regex/xpath and transformations for extracting various required fields
    author: {
      xpath: 'tbody/tr[1]/td/div/small/text()',
      transform: extract(/From: (.*) <.*@.*>/)
    },
    title: {
      xpath: 'tbody/tr[1]/td/div/div[1]/b/a/text()'
    },
    date: {
      xpath: 'tbody/tr[1]/td/div/small/text()',
      transform: extract(/- (.*-.*-.*) /)
    },
    url: {
      xpath: 'tbody/tr[1]/td/div/div[1]/b/a/@href',
      transform: prepend('https://sourceforge.net')
    }
  }, // end of mailing list profile
  // next website/URL (forum)
  'FlightGear forum': {
    enabled: true,
    type: 'archive',
    event: 'document.onmouseup', // when to invoke the event handler
    event_handler: null, // the event handler to be invoked
    url_reg: /https:\/\/forum\.flightgear\.org\/.*/,
    content: {
      selection: getSelectedHtml,
      idStyle: /p[0-9]{6}/,
      parentTag: [
        'className',
        'content',
        'postbody'
      ],
      transform: [
        removeComments,
        forum_quote2cquote,
        forum_smilies2text,
        forum_fontstyle2wikistyle,
        forum_code2syntaxhighlight,
        img2link,
        a2wikilink,
        vid2wiki,
        list2wiki,
        forum_br2newline
      ]
    },
    // vector with tests to be executed for sanity checks (unit testing)
    // postings will be downloaded using the URL specified, and then the author/title 
    // fields extracted using the outer regex and matched against what is expected
    tests: [
      {
        url: 'https://forum.flightgear.org/viewtopic.php?f=18&p=284108#p284108',
        author: 'mickybadia',
        title: 'OSM still PNG maps'
      },
      {
        url: 'https://forum.flightgear.org/viewtopic.php?f=19&p=284120#p284120',
        author: 'Thorsten',
        title: 'Re: FlightGear\'s Screenshot Of The Month MAY 2016'
      },
       {
        url: 'https://forum.flightgear.org/viewtopic.php?f=71&t=29279&p=283455#p283446',
        author: 'Hooray',
        title: 'Re: Best way to learn Canvas?'
      },
      {
        url: 'https://forum.flightgear.org/viewtopic.php?f=4&t=1460&p=283994#p283994',
        author: 'bugman',
        title: 'Re: eurofighter typhoon'
      } // add other tests below

    ], // end of vector with self-tests
    author: {
      xpath: 'div/div[1]/p/strong/a/text()'
    },
    title: {
      xpath: 'div/div[1]/h3/a/text()'
    },
    date: {
      xpath: 'div/div[1]/p/text()[2]',
      transform: extract(/» (.*?[0-9]{4})/)
    },
    url: {
      xpath: 'div/div[1]/p/a/@href',
      transform: [
        extract(/\.(.*)/),
        prepend('https://forum.flightgear.org')
      ]
    }
  }
};

// hash to map URLs (wiki article, issue tracker, sourceforge link, forum thread etc) to existing wiki templates
var URL2TemplateTable = {
  // placeholder for now
}; // TemplateTable
var EventHandlers = {
  updateTarget: function () {
    alert('not yet implement');
  },
  updateFormat: function () {
    alert('not yet implement');
  }
}; // EventHandlers
// output methods (alert and jQuery for now)
var OUTPUT = {
  // Shows a window.prompt() message box
  msgbox: function (msg) {
    window.prompt('Copy to clipboard ' + Host.getScriptVersion(), msg);
    Host.setClipboard(msg);
  },
  // Show a jQuery dialog
  jQueryDiag: function (msg) {
    // WIP: add separate Target/Format combo boxes for changing the template to be used (e.g. for refs instead of quotes)
    var target_format = '<form name=\'target\'>Target: <select name=\'selection\' onchange=\'EventHandlers.updateTarget();\'><option value=\'0\'>to wiki</option><option value=\'1\'>to forum</option></select>Format: <select name=\'format\' onchange=\'EventHandlers.updateFormat();\'><option value=\'0\'>refonly</option><option value=\'1\'>fgcquote</option></select></form>';
    //var style='background-image: url(' + GM_getResourceURL ('myLogo')+ '); background-attachment: local; background-position: center;  background-repeat: no-repeat; background-size: 70%; opacity: 1.0;'
    var diagDiv = $('<div id="MyDialog"><textarea id="quotedtext" rows="10"cols="80" style=" width: 320px; height: 320px">' + msg + '</textarea>' + target_format + '</div>');
   
    var diagParam = {
      title: 'Copy your quote with Ctrl+c ' + Host.getScriptVersion(),
      modal: true,
      width: 'auto',
      buttons: [
        /*
        {
          text: 'Check',
          click: selfCheckDialog
        },
        */
        /*
        {
          text: 'Setup',
          click: setupDialog
        },
        */
        {
          text: 'Select all',
          click: function () {
            Host.setClipboard(msg);
            $('#quotedtext').select();
          }
        },
        {
          text: 'OK',
          click: function () {
            Host.setClipboard(msg);
            $(this).dialog('close');
          }
        }
      ]
    };
    diagDiv.dialog(diagParam);
  }
};

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

var speechTransformations = [
// ordering is crucial here (most specific first, least specific/most generic last)
  {query:/I have done/gi, replacement:'$author has done'},
  {query:/I\'ve done/gi, replacement:'$author has done'}, //FIXME. queries should really be vectors ...
  
  {query:/I have got/gi, replacement:'$author has got'},
  {query:/I\'ve got/gi, replacement:'$author has got'},
  
  
  {query:/I myself/gi, replacement:'$author himself'},
  {query:/I am/gi, replacement:' $author is'},
  {query:/I can/gi, replacement:'$author can'},
  {query:/I have/gi, replacement:'$author has'},
  {query:/I should/g, replacement:'$author should'},
  {query:/I shall/gi, replacement:'$author shall'},
  {query:/I may/gi, replacement:'$author may'},
  {query:/I will/gi, replacement:'$author will'},
  {query:/I would/gi, replacement:'$author would'},
  {query:/by myself/gi, replacement:'by $author'},
  {query:/and I/gi, replacement:'and $author'},
  {query:/and me/gi, replacement:'and $author'},
  {query:/and myself/gi, replacement:'and $author'},
  
  
  // least specific stuff last (broad/generic stuff is kept as is, with author clarification added in parentheses)
  {query:/ I /, replacement:'I ($author)'},
  {query:/ me /, replacement:'me ($author)'},
  {query:/ my /, replacement:'my ($author)'},
  {query:/myself/, replacement:'myself ($author)'},
  {query:/mine/, replacement:'$author'}
  
];

// try to assist in transforming speech using the transformation vector passed in
// still needs to be exposed via the UI
function transformSpeech(text, author, gender, transformations) {
  // WIP: foreach transformation in vector, replace the search pattern with the matched string (replacing author/gender as applicable)
  for(var i=0;i< transformations.length; i++) {
    var token = transformations[i];
    // patch the replacement string using the correct author name 
    var replacement = token.replacement.replace(/\$author/gi, author);
    text = text.replace(token.query, replacement);
  } // end of token transformation
  // console.log("transformed text is:"+text);
  return text;
} // transformSpeech

// run a self-test
(function() {
var author ="John Doe";
var transformed = transformSpeech("I have decided to commit a new feature", author, null, speechTransformations );
if (transformed !== author+" has decided to commit a new feature")
  Host.dbLog("FIXME: Speech transformations are not working correctly");
}) ();
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

var MONTHS = [
  'Jan',
  'Feb',
  'Mar',
  'Apr',
  'May',
  'Jun',
  'Jul',
  'Aug',
  'Sep',
  'Oct',
  'Nov',
  'Dec'
];
// Conversion for forum emoticons
var EMOTICONS = [
  [/:shock:/g,
  'O_O'],
  [
    /:lol:/g,
    '(lol)'
  ],
  [
    /:oops:/g,
    ':$'
  ],
  [
    /:cry:/g,
    ';('
  ],
  [
    /:evil:/g,
    '>:)'
  ],
  [
    /:twisted:/g,
    '3:)'
  ],
  [
    /:roll:/g,
    '(eye roll)'
  ],
  [
    /:wink:/g,
    ';)'
  ],
  [
    /:!:/g,
    '(!)'
  ],
  [
    /:\?:/g,
    '(?)'
  ],
  [
    /:idea:/g,
    '(idea)'
  ],
  [
    /:arrow:/g,
    '(->)'
  ],
  [
    /:mrgreen:/g,
    'xD'
  ]
];
// ##################
// # Main functions #
// ##################
window.addEventListener('load', init);
Host.dbLog('matched page, load handler registered');

// Initialize (matching page loaded)
function init() {
  Host.dbLog('page load handler invoked');
  var profile = getProfile();
  
  Host.dbLog("Profile type is:"+profile.type);
  
  // Dispatch to correct event handler (depending on website/URL)
  // TODO: this stuff could/should be moved into the config hash itself
  
  if (profile.type=='wiki') {
    profile.event_handler(); // just for testing
    return;
  }
   
    Host.dbLog('using default mode');
    //document.onmouseup = instantCquote;
    // preparations for moving the the event/handler logic also into the profile hash, so that the wiki (edit mode) can be handled equally
    eval(profile.event+"=instantCquote");
    
  
} // init()


// The main function
function instantCquote() {
  var profile = getProfile();
  
  // TODO: use config hash here
  var selection = document.getSelection(),
  output = {
  },
  field = {
  },
  post_id=0;
  
  try {
    post_id = getPostId(selection, profile);
  } 
  catch (error) {
    Host.dbLog('Failed extracting post id\nProfile:' + profile);
    return;
  }
  if (selection.toString() === '') {
    Host.dbLog('No text is selected, aborting function');
    return;
  }
  if (!checkValid(selection, profile)) {
    Host.dbLog('Selection is not valid, aborting function');
    return;
  }  // TODO: this needs to be refactored so that it can be also reused by the async/AJAX mode
  // to extract fields in the background (i.e. move to a separate function)

  Host.dbLog("Starting extraction/transformation loop");
  for (field in profile) {
    if (field === 'name') continue;
    if (field ==='type' || field === 'event' || field === 'event_handler') continue; // skip fields that don't contain xpath expressions
    Host.dbLog("Extracting field using field id:"+post_id);
    var fieldData = extractFieldInfo(profile, post_id, field);
    var transform = profile[field].transform;
    if (transform !== undefined) {
      Host.dbLog('Field \'' + field + '\' before transformation:\n\'' + fieldData + '\'');
      fieldData = applyTransformations(fieldData, transform);
      Host.dbLog('Field \'' + field + '\' after transformation:\n\'' + fieldData + '\'');
    }
    output[field] = fieldData;
  } // extract and transform all fields for the current profile (website)
  Host.dbLog("extraction and transformation loop finished");
  output.content = stripWhitespace(output.content);
  output = createCquote(output);
  outputText(output);
}



function getPostingDataAJAX(profile, url) {
  Host.dbLog("Fetching posting via AJAX helper: "+url);
  Host.dbLog("Source profile is="+profile);
  var expr=CONFIG[profile].title.xpath;
  Host.dbLog("xpath expression is:"+expr);
  
  Host.download(url, function (response) {
    Host.dbLog("Download status:"+response.statusText);
    var blob = response.responseText;
    
    var doc = Host.make_doc(blob,'text/html'); //new DOMParser().parseFromString(blob,'text/html');
    // var xpath = '//*[@id="' + id + '"]/' + profile[field].xpath;
    var xpath = '//'+expr; // this is simplified, because the real regex is more complex (see above), but we don't have the post ID when downloading postings via AJAX
    var result = Host.eval_xpath(doc,xpath).stringValue; // doc.evaluate(xpath, doc, null, XPathResult.STRING_TYPE, null);
    Host.dbLog("title:"+result); 
      
    }); // AJAX callback
  
  
} // getPostingDataAJAX


function runProfileTests() {
  
  for (var profile in CONFIG) {
    if (CONFIG[profile].type != 'archive') continue; // skip the wiki entry, because it's not an actual archive that we need to test
    
    for (var test in CONFIG[profile].tests) {
      var required_data = CONFIG[profile].tests[test];
      var title = required_data.title;
      //dbLog('Running test for posting titled:' + title);
      // fetch posting via getPostingDataAJAX() and compare to the fields we are looking for (author, title, date)
      getPostingDataAJAX(profile, required_data.url);
    } // foreach test

  } // foreach profile (website)
  
} //runProfileTests

function selfCheckDialog() {
  var sections = '<h3>Important APIs:</h3><div id="api_checks"><font color="red">(to be added here using the Environment.runAPITests() helper)</font></div>';

  
  

  try {
   runProfileTests.call(undefined); // check website profiles
  }
  catch (e) {
      alert(e.message);
  }
  
  for (var profile in CONFIG) {
    // TODO: also check if enabled or not
    if (CONFIG[profile].type != 'archive') continue; // skip the wiki entry, because it's not an actual archive that we need to test
    var test_results = '';
    for (var test in CONFIG[profile].tests) {
      // var fieldData = extractFieldInfo(profile, post_id, 'author');
      test_results += CONFIG[profile].tests[test].title + '<p/>';
    }
    sections +='<h3>' + profile + ':<font color="blue">'+ CONFIG[profile].url_reg+'</font></h3><div><p>' + test_results + '</p></div>\n';
  }  // https://jqueryui.com/accordion/
  
 
  var checkDlg = $('<div id="selfCheck" title="Self Check dialog"><p><div id="accordion">' + sections + '</div></p></div>');
  
   // run all API tests, invoke the callback to obtain the status
  Environment.runAPITests(Host, function(meta) {
  
  //console.log('Running API test '+meta.name);
    
  meta.test(function(result) {
   var status = (result)?'success':'fail';
   var test = $("<p></p>").text('Running API test '+meta.name+':'+status); ;
   $('#api_checks', checkDlg).append(test);
  });
    
  });
 
  
  //$('#accordion',checkDlg).accordion();
  checkDlg.dialog({
    width: 700,
    height: 500,
    open: function () {
      // http://stackoverflow.com/questions/2929487/putting-a-jquery-ui-accordion-in-a-jquery-ui-dialog
      $('#accordion').accordion({
        autoHeight: true
      });
    }
  }); // show dialog
} // selfCheckDialog


// show a simple configuration dialog (WIP)
function setupDialog() {
  //alert("configuration dialog is not yet implemented");
  var checked = (Host.get_persistent('debug_mode_enabled', false) === true) ? 'checked' : '';
  //dbLog("value is:"+get_persistent("debug_mode_enabled"));
  //dbLog("persistent debug flag is:"+checked);
  var setupDiv = $('<div id="setupDialog" title="Setup dialog">NOTE: this configuration dialog is still work-in-progress</p><label><input id="debugcb" type="checkbox"' + checked + '>Enable Debug mode</label><p/><div id="progressbar"></div></div>');
  setupDiv.click(function () {
    //alert("changing persistent debug state");
    Host.set_persistent('debug_mode_enabled', $('#debugcb').is(':checked'));
  });
  //MediaWiki editing stub, based on: https://www.mediawiki.org/wiki/API:Edit#Editing_via_Ajax
  //only added here to show some status info in the setup dialog
  Host.download('http://wiki.flightgear.org/api.php?action=query&prop=info|revisions&intoken=edit&rvprop=timestamp&titles=Main%20Page', function (response) {
    var message = 'FlightGear wiki login status (AJAX):';
    var status = response.statusText;
    var color = (status == 'OK') ? 'green' : 'red';
    Host.dbLog(message + status);
    var statusDiv = $('<p>' + message + status + '</p>').css('color', color);
    setupDiv.append(statusDiv);
  });
  setupDiv.dialog();
} // setupDialog


// this  can be used to download/cache $FG_ROOT/options.xml so that fgfs CLI arguments can be recognized automatically
// which can help transforming postings correctly
function downloadOptionsXML() {

  // download $FG_ROOT/options.xml
          Host.download("https://sourceforge.net/p/flightgear/fgdata/ci/next/tree/options.xml?format=raw", function(response) {
            var xml = response.responseText;
            var doc = Host.make_doc(xml, 'text/xml');
            // https://developer.mozilla.org/en-US/docs/Web/API/XPathResult
            var options = Host.eval_xpath(doc, '//*/option', XPathResult.ORDERED_NODE_SNAPSHOT_TYPE);
            
            // http://help.dottoro.com/ljgnejkp.php
            Host.dbLog("Number of options found in options.xml:"+options.snapshotLength);
            
            // http://help.dottoro.com/ljtfvvpx.php
            
              // https://sourceforge.net/p/flightgear/fgdata/ci/next/tree/options.xml
              
            
          }); // end of options.xml download

  
} // downloadOptionsXML

function getProfile(url=undefined) {
  
  if(url === undefined) 
    url=window.location.href;
  else
    url=url;
  
  Host.dbLog("getProfile call URL is:"+url);
  
  for (var profile in CONFIG) {
    if (url.match(CONFIG[profile].url_reg) !== null) {
      Host.dbLog('Matching website profile found');
      var invocations = Host.get_persistent(Host.getScriptVersion(), 0);
      Host.dbLog('Number of script invocations for version ' + Host.getScriptVersion() + ' is:' + invocations);

      // determine if we want to show a config dialog
      if (invocations === 0) {
        Host.dbLog("ask for config dialog to be shown");
        var response = confirm('This is your first time running version ' + Host.getScriptVersion() + '\nConfigure now?');
        if (response) {
                  
          // show configuration dialog (jQuery)
          setupDialog();
        } 
        else {
        } // don't configure

      }      // increment number of invocations, use the script's version number as the key, to prevent the config dialog from showing up again (except for updated scripts)

      Host.dbLog("increment number of script invocations");
      Host.set_persistent(Host.getScriptVersion(), invocations + 1);
      return CONFIG[profile];
    }
    Host.dbLog('Could not find matching URL in getProfile() call!');
  }
}// Get the HTML code that is selected

function getSelectedHtml() {
  // From http://stackoverflow.com/a/6668159
  var html = '',
  selection = document.getSelection();
  if (selection.rangeCount) {
    var container = document.createElement('div');
    for (var i = 0; i < selection.rangeCount; i++) {
      container.appendChild(selection.getRangeAt(i).cloneContents());
    }
    html = container.innerHTML;
  }
  Host.dbLog('instantCquote(): Unprocessed HTML\n\'' + html + '\'');
  return html;
}// Gets the selected text

function getSelectedText() {
  return document.getSelection().toString();
}// Get the ID of the post
// (this needs some work so that it can be used by the AJAX mode, without an actual selection)

function getPostId(selection, profile, focus) {
  if (focus !== undefined) {
    Host.dbLog("Trying to get PostId with defined focus");
    selection = selection.focusNode.parentNode;
  } else {
    Host.dbLog("Trying to get PostId with undefined focus");
    selection = selection.anchorNode.parentNode;
  }
  while (selection.id.match(profile.content.idStyle) === null) {
    selection = selection.parentNode;
  }
  Host.dbLog("Selection id is:"+selection.id);
  return selection.id;
}

// Checks that the selection is valid
function checkValid(selection, profile) {
  var ret = true,
  selection_cp = {
  },
  tags = profile.content.parentTag;
  for (var n = 0; n < 2; n++) {
    if (n === 0) {
      selection_cp = selection.anchorNode.parentNode;
    } else {
      selection_cp = selection.focusNode.parentNode;
    }
    while (true) {
      if (selection_cp.tagName === 'BODY') {
        ret = false;
        break;
      } else {
        var cont = false;
        for (var i = 0; i < tags.length; i++) {
          if (selection_cp[tags[0]] === tags[i]) {
            cont = true;
            break;
          }
        }
        if (cont) {
          break;
        } else {
          selection_cp = selection_cp.parentNode;
        }
      }
    }
  }
  ret = ret && (getPostId(selection, profile) === getPostId(selection, profile, 1));
  return ret;
}// Extracts the raw text from a certain place, using an XPath

function extractFieldInfo(profile, id, field) {
  
  if (field === 'content') {
    Host.dbLog("Returning content (selection)");
    return profile[field].selection();
  } else {
    Host.dbLog("Extracting field via xpath:"+field);
    var xpath = '//*[@id="' + id + '"]/' + profile[field].xpath;
    return Host.eval_xpath(document, xpath).stringValue; // document.evaluate(xpath, document, null, XPathResult.STRING_TYPE, null).stringValue;
  }
}// Change the text using specified transformations

function applyTransformations(fieldInfo, trans) {
  if (typeof trans === 'function') {
    return trans(fieldInfo);
  } else if (Array.isArray(trans)) {
    for (var i = 0; i < trans.length; i++) {
      fieldInfo = trans[i](fieldInfo);
      Host.dbLog('applyTransformations(): Multiple transformation, transformation after loop #' + (i + 1) + ':\n\'' + fieldInfo + '\'');
    }
    return fieldInfo;
  }
}// Formats the quote

function createCquote(data, light_quotes = true) {
  // skip FGCquote (experimental)
  if (light_quotes) return nonQuotedRef(data);
  var date_added = new Date();
  var wikiText = '{{FGCquote\n' + ((data.content.match(/^\s*?{{cquote/) === null) ? '|1= ' : '| ') + data.content + '\n' +
  '|2= ' + createCiteWeb(data) + '\n' +
  '}}';
  return wikiText;
}
function nonQuotedRef(data) {
  return addContentBlob(data) + createRefCite(data);
}// 

function addContentBlob(data) {
  return data.content;
}// wrap citation in ref tags

function createRefCite(data) {
  return '<ref>' + createCiteWeb(data) + '</ref>';
}
function createCiteWeb(data) {
  var date_added = new Date();
  var wikiText = '{{cite web\n' +
  '  |url    = ' + data.url + '\n' +
  '  |title  = ' + nowiki(data.title) + '\n' +
  '  |author = ' + nowiki(data.author) + '\n' +
  '  |date   = ' + datef(data.date) + '\n' +
  '  |added  = ' + datef(date_added.toDateString()) + '\n' +
  '  |script_version = ' + GM_info.script.version + '\n' +
  '  }}\n';
  return wikiText;
}// Output the text.
// Tries the jQuery dialog, and falls back to window.prompt()

function outputText(msg) {
  try {
    OUTPUT.jQueryDiag(msg);
    // TODO: unify code & call Host.setClipboard() here
  } 
  catch (err) {
    msg = msg.replace(/&lt;\/syntaxhighligh(.)>/g, '</syntaxhighligh$1');
    OUTPUT.msgbox(msg);
  }
}// #############
// # Utilities #
// #############

function extract(regex) {
  return function (text) {
    return text.match(regex) [1];
  };
}
function prepend(prefix) {
  return function (text) {
    return prefix + text;
  };
}
function removeComments(html) {
  return html.replace(/<!--.*?-->/g, '');
}// Not currently used (as of June 2015), but kept just in case


// currently unused
function escapePipes(html) {
  html = html.replace(/\|\|/g, '{{!!}}');
  html = html.replace(/\|\-/g, '{{!-}}');
  return html.replace(/\|/g, '{{!}}');
}// Converts HTML <a href="...">...</a> tags to wiki links, internal if possible.

function a2wikilink(html) {
  // Links to wiki images, because
  // they need special treatment, or else they get displayed.
  html = html.replace(/<a.*?href="http:\/\/wiki\.flightgear\.org\/File:(.*?)".*?>(.*?)<\/a>/g, '[[Media:$1|$2]]');
  // Wiki links without custom text.
  html = html.replace(/<a.*?href="http:\/\/wiki\.flightgear\.org\/(.*?)".*?>http:\/\/wiki\.flightgear\.org\/.*?<\/a>/g, '[[$1]]');
  // Links to the wiki with custom text
  html = html.replace(/<a.*?href="http:\/\/wiki\.flightgear\.org\/(.*?)".*?>(.*?)<\/a>/g, '[[$1|$2]]');
  // Remove underscores from all wiki links
  var list = html.match(/\[\[.*?\]\]/g);
  if (list !== null) {
    for (var i = 0; i < list.length; i++) {
      html = html.replace(list[i], underscore2Space(list[i]));
    }
  }  // Convert non-wiki links
  // TODO: identify forum/devel list links, and use the AJAX/Host.download helper to get a title/subject for unnamed links (using the existing xpath/regex helpers for that)

  html = html.replace(/<a.*?href="(.*?)".*?>(.*?)<\/a>/g, '[$1 $2]');
  // Remove triple dots from external links.
  // Replace with raw URL (MediaWiki converts it to a link).
  list = html.match(/\[.*?(\.\.\.).*?\]/g);
  if (list !== null) {
    for (var i = 0; i < list.length; i++) {
      html = html.replace(list[i], list[i].match(/\[(.*?) .*?\]/) [1]);
    }
  }
  return html;
}// Converts images, including images in <a> links

function img2link(html) {
  html = html.replace(/<a[^<]*?href="([^<]*?)"[^<]*?><img.*?src="http:\/\/wiki\.flightgear\.org\/images\/.*?\/.*?\/(.*?)".*?><\/a>/g, '[[File:$2|250px|link=$1]]');
  html = html.replace(/<img.*?src="http:\/\/wiki\.flightgear\.org\/images\/.*?\/.*?\/(.*?)".*?>/g, '[[File:$1|250px]]');
  html = html.replace(/<a[^<]*?href="([^<]*?)"[^<]*?><img.*?src="(.*?)".*?><\/a>/g, '(see [$2 image], links to [$1 here])');
  return html.replace(/<img.*?src="(.*?)".*?>/g, '(see the [$1 linked image])');
}// Converts smilies

function forum_smilies2text(html) {
  html = html.replace(/<img src="\.\/images\/smilies\/icon_.*?\.gif" alt="(.*?)".*?>/g, '$1');
  for (var i = 0; i < EMOTICONS.length; i++) {
    html = html.replace(EMOTICONS[i][0], EMOTICONS[i][1]);
  }
  return html;
}// Converts font formatting

function forum_fontstyle2wikistyle(html) {
  html = html.replace(/<span style="font-weight: bold">(.*?)<\/span>/g, '\'\'\'$1\'\'\'');
  html = html.replace(/<span style="text-decoration: underline">(.*?)<\/span>/g, '<u>$1</u>');
  html = html.replace(/<span style="font-style: italic">(.*?)<\/span>/g, '\'\'$1\'\'');
  return html.replace(/<span class="posthilit">(.*?)<\/span>/g, '$1');
}// Converts code blocks

function forum_code2syntaxhighlight(html) {
  var list = html.match(/<dl class="codebox">.*?<code>(.*?)<\/code>.*?<\/dl>/g),
  data = [
  ];
  if (list === null) return html;
  for (var n = 0; n < list.length; n++) {
    data = html.match(/<dl class="codebox">.*?<code>(.*?)<\/code>.*?<\/dl>/);
    html = html.replace(data[0], processCode(data));
  }
  return html;
}// Strips any whitespace from the beginning and end of a string

function stripWhitespace(html) {
  html = html.replace(/^\s*?(\S)/, '$1');
  return html.replace(/(\S)\s*?\z/, '$1');
}// Process code, including basic detection of language

function processCode(data) {
  var lang = '',
  code = data[1];
  code = code.replace(/&nbsp;/g, ' ');
  if (code.match(/=?.*?\(?.*?\)?;/) !== null) lang = 'nasal';
  if (code.match(/&lt;.*?&gt;.*?&lt;\/.*?&gt;/) !== null || code.match(/&lt;!--.*?--&gt;/) !== null) lang = 'xml';
  code = code.replace(/<br\/?>/g, '\n');
  return '<syntaxhighlight lang="' + lang + '" enclose="div">\n' + code + '\n&lt;/syntaxhighlight>';
}// Converts quote blocks to Cquotes

function forum_quote2cquote(html) {
  html = html.replace(/<blockquote class="uncited"><div>(.*?)<\/div><\/blockquote>/g, '{{cquote|$1}}');
  if (html.match(/<blockquote>/g) === null) return html;
  var numQuotes = html.match(/<blockquote>/g).length;
  for (var n = 0; n < numQuotes; n++) {
    html = html.replace(/<blockquote><div><cite>(.*?) wrote.*?:<\/cite>(.*?)<\/div><\/blockquote>/, '{{cquote|$2|$1}}');
  }
  return html;
}// Converts videos to wiki style

function vid2wiki(html) {
  // YouTube
  html = html.replace(/<div class="video-wrapper">\s.*?<div class="video-container">\s*?<iframe class="youtube-player".*?width="(.*?)" height="(.*?)" src="http:\/\/www\.youtube\.com\/embed\/(.*?)".*?><\/iframe>\s*?<\/div>\s*?<\/div>/g, '{{#ev:youtube|$3|$1x$2}}');
  // Vimeo
  html = html.replace(/<iframe src="http:\/\/player\.vimeo\.com\/video\/(.*?)\?.*?" width="(.*?)" height="(.*?)".*?>.*?<\/iframe>/g, '{{#ev:vimeo|$1|$2x$3}}');
  return html.replace(/\[.*? Watch on Vimeo\]/g, '');
}// Not currently used (as of June 2015), but kept just in case

// currently unused
function escapeEquals(html) {
  return html.replace(/=/g, '{{=}}');
}// <br> to newline.

function forum_br2newline(html) {
  html = html.replace(/<br\/?><br\/?>/g, '\n');
  return html.replace(/<br\/?>/g, '\n\n');
}// Forum list to wiki style

function list2wiki(html) {
  var list = html.match(/<ul>(.*?)<\/ul>/g);
  if (list !== null) {
    for (var i = 0; i < list.length; i++) {
      html = html.replace(/<li>(.*?)<\/li>/g, '* $1\n');
    }
  }
  list = html.match(/<ol.*?>(.*?)<\/ol>/g);
  if (list !== null) {
    for (var i = 0; i < list.length; i++) {
      html = html.replace(/<li>(.*?)<\/li>/g, '# $1\n');
    }
  }
  html = html.replace(/<\/?[uo]l>/g, '');
  return html;
}
function nowiki(text) {
  return '<nowiki>' + text + '</nowiki>';
}// Returns the correct ordinal adjective

function ordAdj(date) {
  date = date.toString();
  if (date == '11' || date == '12' || date == '13') {
    return 'th';
  } else if (date.substr(1) == '1' || date == '1') {
    return 'st';
  } else if (date.substr(1) == '2' || date == '2') {
    return 'nd';
  } else if (date.substr(1) == '3' || date == '3') {
    return 'rd';
  } else {
    return 'th';
  }
}

// Formats the date to this format: Apr 26th, 2015
function datef(text) {
  var date = new Date(text);
  return MONTHS[date.getMonth()] + ' ' + date.getDate() + ordAdj(date.getDate()) + ', ' + date.getFullYear();
}
function underscore2Space(str) {
  return str.replace(/_/g, ' ');
}
References
  1. ↑ John Denker (Jul 16th, 2007). [Flightgear-devel] development process (was: chaos...).
  2. ↑ Curtis Olson (Jul 27th, 2011). Re: [Flightgear-devel] The state of things in Flight Gear.
  3. ↑ Hal V. Engel (Jul 27th, 2011). Re: [Flightgear-devel] The state of things in Flight Gear.
  4. ↑ Curtis Olson (Jul 27th, 2011). Re: [Flightgear-devel] The state of things in Flight Gear.
  5. ↑ Thorsten (Mar 21st, 2016). Re: quoting on the wiki.
  6. ↑ James Turner (Mon, 28 Jul 2008 10:06:05 -0700). [Flightgear-devel] Project tracking.
  7. ↑ bugman (Apr 17th, 2016). Re: What is the QT launcher?.
  8. ↑ James Turner (Mar 21st, 2016). Re: [Flightgear-devel] Wiki Quotes.
  9. ↑ Stuart Buchanan (Mar 19th, 2016). Re: [Flightgear-devel] Wiki Quotes.
Retrieved from "https://wiki.flightgear.org/w/index.php?title=FlightGear_wiki:Instant-Refs&oldid=97906"