Difference between revisions of "FlightGear wiki:Instant-Refs"

From FlightGear wiki
Jump to: navigation, search
(Converted the usage of the {{forumref}} template to {{forum link}}.)
 
(343 intermediate revisions by 8 users not shown)
Line 1: Line 1:
{{Note|FlightGear development is not centrally coordinated in any way - at the very best, FlightGear development is "self-coordinated" - i.e. contributors discuss ideas and make proposals to contribute in a certain fashion and then team up to implement certain features and building blocks temporarily.  
+
[[File:Quotes-logo-200x200.png|thumb]]
 +
[[File:Instant-cquotes-firefox.png|thumb|Instant-Cquotes user script in Firefox]]
 +
[[File:ContentMonster-web-extension.png|thumb|Screenshot showing the web-extension port of the '''InstantRefs''' user-script in FireFox.]]
  
Obviously, ideas and feature requests made by end-users are also appreciated. But at the end of the day, people who do the actual work have obviously more say about future development than those who merely make suggestions. And contributors tend to prioritize work on items that are prioritized/suggested by other contributors, especially those offering to get involved and help in some way, and of those having some kind of track record in the form of past contributions.
+
[[File:Ref-only-quotes.png|thumb|Instant-Cquotes screenshot prototyping runtime format selection]]
  
But given the lack of development manpower, many good ideas tend to have a fairly long shelf life unfortunately. So that good ideas may be forgotten over time.
+
The '''Instant-Refs''' (renamed in 11/2016, originally, Instant-cquotes) script is a browser addon (as of late 2017, a so called web-extension) implemented in JavaScript that started out as a user script in order to convert excerpts (created via copy&paste) from FlightGear forum or [[mailing list]] postings into MediaWiki markup/quotes to be used on the FlightGear wiki.
  
This is also why it is important for other developers/contributors to know who came up with a certain idea and who originally supported/supports it, possibly months (or even years) after a discussion took place - which is why good ideas should not just be preserved, but accompanied by corresponding quotes, linking back to the original discussion, so that potential contributors can make up their own minds to determine if/how they want to get involved in some effort or not. The script that you can find below is intended to help with this (as well as allowing people to easily reuse forum/devel list announcements in wiki articles, e.g. to update the changelog, newsletter or "release plan/lessons learnt" section) - it allows people to easily copy&paste important discussions over to the wiki, without having to rewrite any text, and without having to put together proper cquotes manually. In fact, it's a matter of just a few seconds.
+
It is being developed and maintained by a group of volunteers involved in maintaining the wiki and in trying to provide more up-to-date information to end-users who may not be as involved in the various FlightGear-related communication channels.
Use this article's discussion page to get in touch (e.g. if there are any bugs/problems or features requests).}}
+
 
 +
== Background and motivation ==
 +
FlightGear's development is, at best, "self-coordinated", meaning that contributors discuss ideas and make proposals to contribute in a certain fashion and then team up to implement certain features and building blocks, often just temporarily.
 +
 
 +
Unfortunately, due to a lack of development manpower, many ideas are not implemented immediately; it is, thus, important to know their pros and cons, as well as who originally proposed them and/or might help with their implementation, even after a long time, preferably with links to the original discussions so that new contributors can decide whether to get involved in some effort or not. The project documentation, however, is in great need of improvement<ref>{{cite web
 +
| url    = http://sourceforge.net/p/flightgear/mailman/message/15444440/
 +
| title  = <nowiki>[Flightgear-devel] development process (was:  chaos...)</nowiki>
 +
| author = <nowiki>John Denker</nowiki>
 +
| date  = Jul 16th, 2007
 +
| added  = Jul 16th, 2007
 +
| script_version = 0.23
 +
}}</ref> and usually significantly lacking behind:<ref name="OlsonStateThingsFG">{{cite web
 +
| url    = http://sourceforge.net/p/flightgear/mailman/message/27861667/
 +
| title  = <nowiki>Re: [Flightgear-devel] The state of things in Flight Gear</nowiki>
 +
| author = <nowiki>Curtis Olson</nowiki>
 +
| date  = Jul 27th, 2011
 +
| added  = Jul 27th, 2011
 +
| script_version = 0.23
 +
}}</ref> many core developers update it seldomly, if not anymore, as it takes time to write it, as well as an understanding of the inner workings of a complex system such as FlightGear. Furthermore, this task might not be attractive due to its short term impact on the project,<ref>{{cite web
 +
| url    = http://sourceforge.net/p/flightgear/mailman/message/27861562/
 +
| title  = <nowiki>Re: [Flightgear-devel] The state of things in Flight Gear</nowiki>
 +
| author = <nowiki>Hal V. Engel</nowiki>
 +
| date  = Jul 27th, 2011
 +
| added  = Jul 27th, 2011
 +
| script_version = 0.23
 +
}}</ref> and it is overwhelming to think about creating documentation that would address the needs of many different kinds of contributors with different backgrounds, experience levels and goals.<ref name="OlsonStateThingsFG" />
 +
 
 +
Forum and mailing lists discussions have therefore become the only up-to-date (albeit difficult to filter)<ref>{{cite web
 +
| url    = http://forum.flightgear.org/viewtopic.php?p=280058#p280058
 +
| title  = <nowiki>Re: quoting on the wiki</nowiki>
 +
| author = <nowiki>Thorsten</nowiki>
 +
| date  = Mar 21st, 2016
 +
| added  = Mar 21st, 2016
 +
| script_version = 0.25
 +
}}</ref> source of information about recent development progress; this makes it tricky to know what is going on, what needs fixing, what were the decisions taken by the developers.<ref>{{cite web
 +
| url    = https://www.mail-archive.com/flightgear-devel%40lists.sourceforge.net/msg17198.html
 +
| title  = <nowiki>[Flightgear-devel] Project tracking</nowiki>
 +
| author = <nowiki>James Turner</nowiki>
 +
| date  = Mon, 28 Jul 2008 10:06:05 -0700
 +
}}</ref>
 +
 
 +
The aim of the '''Instant-Refs''' script is to help wiki editors to copy relevant excerpts from such sources, formatting them as proper [[Template:Cquote|quotations]], and help bootstrap new articles collecting them until a dedicated rewrite is made. It can also be used to reuse announcements to update the changelogs, [[Next newsletter|newsletter]] or the [[Release plan/Lessons learned]] page.
 +
 
 +
After being away from the wiki system for some time it becomes more of an effort to re-learn how to start a new file and figure out how to format it , with what headings, etc. Sometimes it is almost too much to do the original write up of the original post and all the work of that layout and effort to have to also duplicate it in a wiki article. If I was a little more current and affluent with the wiki editor, I might not be so hesitant to create the work there instead of the forum.<ref>{{cite web
 +
  |url    =  https://forum.flightgear.org/viewtopic.php?p=284698#p284698
 +
  |title  =  <nowiki> Re: Breaking down NLCD raster image </nowiki>
 +
  |author =  <nowiki> wlbragg </nowiki>
 +
  |date  =  May 9th, 2016
 +
  |added  =  May 9th, 2016
 +
  |script_version = 0.38
 +
  }}</ref>
 +
 
 +
 
 +
In a few cases, such collections of quotes helped not only create bootstrap new articles, but even actual features, too.
 +
 
 +
In other cases, quotes have been used to update documentation of features whose maintainers may not be actively involved in FlightGear (e.g. [[Rembrandt]]), to help document discussions that are taking place in the meantime, and provide some background information for people interested in the corresponding feature.
 +
 
 +
<!--
 +
== Installation ==
 +
# Install a user script manager. On Firefox, you can use [https://addons.mozilla.org/en-US/firefox/addon/greasemonkey/ Greasemonkey]; on Chrome/Chromium, Opera or Safari, you can use [https://tampermonkey.net/ Tampermonkey] (download links: [https://tampermonkey.net/index.php?ext=dhdg&browser=chrome Chrome/Chromium], [https://tampermonkey.net/index.php?ext=dhdg&browser=opera Opera], [https://tampermonkey.net/index.php?ext=dhdg&browser=safari Safari]).
 +
# Visit [https://greasyfork.org/en/scripts/19331-instant-cquotes the Instant-Refs page on GreasyFork] and click on {{button|Install this script}} (green button). If Greasemonkey/Tampermonkey prompts you to confirm the installation, agree to do so.
 +
 
 +
[[File:Howto-install-instant-cquotes.png|thumb|Click the green button to install the script]]
 +
 
 +
=== Manual installation ===
 +
{{note|This will install the most recent development version of the script, which might contain bugs. Also, GreaseMonkey/TamperMonkey will not update it automatically whenever a new version is released.}}
 +
 
 +
* '''Firefox'''
 +
# Install Greasemonkey.
 +
# Save [[#The Script|the script]] below as <code>instant_cquotes.user.js</code>, then drag-and-drop it into Firefox.
 +
[[File:Greasemonkey-setup-on-firefox.png|thumb|Screenshot showing the Greasemonkey setup dialog (on Firefox)]]
 +
 
 +
* '''Chrome/Chromium''', '''Opera''', or '''Safari'''
 +
# Install Tampermonkey.
 +
# Navigate to '''Add a new Script'''.
 +
# Copy and paste [[#The Script|the script]] below into the editing window.
 +
# Click the {{button|Save}} button (just above the {{button|Search}} button).
 +
-->
 +
=== Configuration ===
 +
[[File:User-script-menu.png|thumb|GreaseMonkey menu shown in FireFox with instanct cquotes menu items]]
 +
 
 +
[[File:Config-dialog-instant-cquotes.png|thumb|The configuration dialog for the Instant-Cquotes script]]
 +
 
 +
As of version 0.30, a dedicated configuration dialog is in the process of being added, so that certain script features can be dynamically configured, without having to edit the script. For now, this is merely a placeholder that provides a checkbox to easily enable/disable the debug mode. In the future, we are hoping to also expose other features this way.
 +
 
 +
== Usage ==
 +
[[File:Updated-cquotes-script-by-redleader.png|thumb|Instant-Cquotes script, with updates contributed by Red Leader]]
 +
 
 +
[[File:New-cquotes.png|thumb|Screenshot showing Instant-Cquotes 0.30 at work]]
 +
 
 +
# Go to some [[mailing list]] archive URL, for example [http://sourceforge.net/p/flightgear/mailman/message/32400727/] or any forum message, such as {{forum link|title=Re: Get objects to show up on Map/Radar|t=23299|anchor=p212558}}.
 +
# Select the relevant portion of text.
 +
# When you release the mouse button, a box will appear containing the converted text (for now, mainly properly-referenced quotes for the wiki).
 +
# The text will be automatically selected and copied to the clipboard.
 +
# Paste the text into the desired wiki page.
 +
 
 +
For example, by selecting part of the forum post in the link above you can get the following quotation:
  
== Example Output ==
 
The output may look like this (click the link to see the original forum posting):
 
 
{{FGCquote
 
{{FGCquote
  |The upcoming FlightGear version (3.2) will contain a canvas-based map dialog, including a modular "plugin" system for creating custom map layers and charts with roughly ~50 lines of code, most of it boilerplate. <br><br>This is entirely XML/Nasal based (scripted) - symbols can be pretty much anything, raster or vector images (png or svg), but even animated. Styling can be customized, too.<br><br>For more info, I suggest to check out:<br><br>[[MapStructure#Porting_the_map_dialog|http://wiki.flightgear.org/MapStructure ... map_dialog]]<br>[[File:MapStructureDialog.png|250px]] <br>
+
|1= The upcoming FlightGear version (3.2) will contain a canvas-based map dialog, including a modular "plugin" system for creating custom map layers and charts with roughly ~50 lines of code, most of it boilerplate.  
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=212558#p212558
+
This is entirely XML/Nasal based (scripted) - symbols can be pretty much anything, raster or vector images (png or svg), but even animated. Styling can be customied, too.
    |title=<nowiki>Re: Get objects to show up on Map/Radar</nowiki>
+
For more info, I suggest to check out:
    |author=<nowiki>Hooray</nowiki>
+
[[MapStructure#Porting the map dialog]]
    |date=<nowiki>Sat Jun 14</nowiki>
+
 
  }}
+
[[File:MapStructureDialog.png|250px]]
 +
|2= {{cite web
 +
  | url   = http://forum.flightgear.org/viewtopic.php?p=212558#p212558
 +
  | title = <nowiki>Re: Get objects to show up on Map/Radar</nowiki>
 +
  | author = <nowiki>Hooray</nowiki>
 +
  | date   = Jun 14th, 2014
 +
  }}
 
}}
 
}}
  
== Background & Installation ==
+
=== On quoting ===
* A simple userscript extension implemented in JavaScript
+
{{Main article|FlightGear wiki:Quoting Guidelines}}
* Supported by FireFox, Chrome, Chromium, probably Opera and others
+
* in case of doubt, just install the GreaseMonkey/[https://chrome.google.com/webstore/detail/tampermonkey/dhdgffkkebhmkfjojejmpbldmpobfkfo?hl=en TamperMonkey] extension
+
* install the script (on firefox, save the script as instant_cquotes.user.js, then drag'n'drop it into firefox).
+
== Usage ==
+
* go to some sourceforge.net mail archive URL, for example: http://sourceforge.net/p/flightgear/mailman/flightgear-devel/thread/5389094A.3080601%40gmail.com/#msg32400727
+
: or to any forum message, such as: http://forum.flightgear.org/viewtopic.php?f=71&t=23299#p212558
+
* copy/mark some relevant portion of text (the script converts links, images and even youtube videos - wiki images are properly converted, too and it will try to retain basic formatting)
+
* and directly get a full cquote paragraph (including author, date, thread and URL) that you can add to the wiki here using CTRL+C & CTRL-V
+
  
== Known Limitations ==
+
Using the Instant-Cquotes script is a good way to bootstrap and write some preliminary notes; however, while quotes might be useful to understand how undocumented subsystems and features work and are definitely better than nothing, they are not meant to replace proper, structured and well-written wiki articles.<ref>{{cite web
* <s>newline2br</s> addNewline() should also insert CR/LF for separating paragraphs in the cquote (also, the trailing slash is not added) {{done}}
+
| url    = http://forum.flightgear.org/viewtopic.php?p=282800#p282800
* quoting code doesn't yet work properly {{Not done}}
+
| title  = <nowiki>Re: What is the QT launcher?</nowiki>
* our regexes may fail once the HTML DOM of the source changes (e.g. phpBB/theme update), so we should better show a warning when that's the case {{Not done}}
+
| author = <nowiki>bugman</nowiki>
* it may make sense to provide multiple regexes that are tried in order {{Not done}}
+
| date  = Apr 17th, 2016
* The script uses a conventional JavaScript alert() box for providing the output, these dialogs are typically restricted to a max size of ~10kb-we may need to explore other/extended options  {{Not done}}
+
| added  = Apr 17th, 2016
* The script has already seen several iterations and created dozens of cquotes we're using in various places, but it might be possible to use the script to update previously created cquotes/FGCquotes automatically, instead of using the getSelection() helper, we could register a '''match''' for wiki.flightgear.org with the '''action=edit''' set, so that we can directly process all text of an edited page (using AJAX calls to open the URL in the background would make sense) {{Not done}}
+
| script_version = 0.25
 +
}}</ref>
  
== Feature Requests & Ideas ==
+
One way to convert pages bootstrapped using quotes is to extract relevant information from them and keep citations only as references; in case important details are missing, they can be asked for on the [[Mailing lists|mailing lists]] (on the forum, the chance to get a complete answer might be lower).<ref>{{cite web
 +
| url    = http://sourceforge.net/p/flightgear/mailman/message/34954385/
 +
| title  = <nowiki>Re: [Flightgear-devel] Wiki Quotes</nowiki>
 +
| author = <nowiki>James Turner</nowiki>
 +
| date  = Mar 21st, 2016
 +
| added  = Mar 21st, 2016
 +
| script_version = 0.25
 +
}}</ref> Another option might be moving the quotes to the Talk page for each entry, which would preserve the sources without clogging up the articles.<ref>{{cite web
 +
| url    = http://sourceforge.net/p/flightgear/mailman/message/34948989/
 +
| title  = <nowiki>Re: [Flightgear-devel] Wiki Quotes</nowiki>
 +
| author = <nowiki>Stuart Buchanan</nowiki>
 +
| date  = Mar 19th, 2016
 +
| added  = Mar 19th, 2016
 +
| script_version = 0.25
 +
}}</ref>
  
 +
As a matter of fact, the whole paragraph above was assembled using this approach; to see for yourself, look up the references at the end of this page. For another example, see [[TerraSync#News]].
  
== The Script ==
+
== Development ==
<syntaxhighlight lang="javascript">// ==UserScript==
+
{{Note|A Chrome/Chromium-specific extension that will not need Tampermonkey installed is under development.}}
// @name      instant-cquotes
+
 
// @description automatically create proper wikimedia cquotes for sourceforge ml and forum text selections
+
=== Resources ===
// @namespace  http://wiki.flightgear.org/
+
* https://www.mediawiki.org/wiki/API:Changing_wiki_content
// @version    0.10
+
* https://www.mediawiki.org/wiki/API:Edit
// @icon      http://wiki.flightgear.org/skins/common/images/icons-fg-135.png
+
 
// @match     http://sourceforge.net/p/flightgear/mailman/*  
+
=== Adding sources ===
// @match     http://forum.flightgear.org/*
+
<!--
// @copyright  2013+, FlightGear.org (bigstones, Philosopher & Hooray)
+
{{WIP}}
// ==/UserScript==
+
-->
+
Adding a new source is pretty straightforward if you understand how xpath and regexes work - basically, you only need an archive (e.g. gmane), and then determine the xpath of each relevant field, as well as the regular expression to process the extracted fields (optional).
var debug = 0;
+
 
+
The basic steps are these:
/* content:
+
# open the user script in an editor
*    - 'selection' supports only getSelectedText, will support getSelectedHtml
+
# navigate to the meta header of the user script
*    - 'transform' takes a "tranform operator", or an ordered array of operators
+
# add a new URL to the top of the script, e.g. by copying/adapting an existing line like this:
* author, title, date, url:
+
<syntaxhighlight lang="javascript">
*    - 'xpath' takes the path to the field, relative to the html parent of the selected text
+
// @match       https://sourceforge.net/p/flightgear/mailman/*
*    - 'transform' takes a "transform operator", or an ordered array of operators
+
</syntaxhighlight>
*/
+
 
var CONFIG = {
+
Once copied, add the new URL:
     "sourceforge.net": {
+
 
        content: {
+
<syntaxhighlight lang="javascript">
            selection: getSelectedText,
+
// @match       http://thread.gmane.org/gmane.games.flightgear.devel/*
            transform: newline2br()
+
</syntaxhighlight>
        },  
+
 
        author: {
+
Next, you need to navigate to the configuration hash to add a new website to it.  
            xpath: "../../../tr/td/div/small/text()",
+
 
            transform: extract(/^From: (.*) <.*@.*>/)
+
Again, it makes sense to simply take an existing configuration hash and adapt it as needed (ignore/omit the tests vector for now by keeping it empty):
        },
+
 
        title: {
+
{{Caution|the following example may meanwhile be outdated, so be sure to look at the actual code instead}}
            xpath: "../../../tr/td/div/div/b/a/text()"
+
 
        },
+
<syntaxhighlight lang="javascript">
        date: {
+
  'Sourceforge Mailing list': {
            xpath: "../../../tr/td/div/small/text()",
+
    enabled: true,
            transform: extract(/ - (.*-.*-.*) /)
+
    type: 'archive',
        },
+
    event: 'document.onmouseup', // when to invoke the event handler
        url: {
+
    event_handler: instantCquote, // the event handler to be invoked
            xpath: "../../../tr/td/div/div/b/a/@href",
+
     url_reg: '^(http|https)://sourceforge.net/p/flightgear/mailman/.*/',
            transform: prepend("http://sourceforge.net")
+
    content: {
        }
+
      xpath: 'tbody/tr[2]/td/pre/text()',
 +
      selection: getSelectedText,
 +
      idStyle: /msg[0-9]{8}/,
 +
      parentTag: [
 +
        'tagName',
 +
        'PRE'
 +
      ],
 +
    transform: [] // vector with transformation callbacks
 +
    }, // content recipe
 +
    // vector with tests to be executed for sanity checks (unit testing)
 +
    tests: [
 +
    ], // end of vector with self-tests
 +
    // regex/xpath and transformations for extracting various required fields
 +
    author: {
 +
      xpath: 'tbody/tr[1]/td/div/small/text()',
 +
      transform: [extract(/From: (.*) <.*@.*>/)]
 +
    },
 +
    title: {
 +
      xpath: 'tbody/tr[1]/td/div/div[1]/b/a/text()'
 +
      transform: []
 +
    },
 +
    date: {
 +
      xpath: 'tbody/tr[1]/td/div/small/text()',
 +
      transform: [extract(/- (.*-.*-.*) /)]
 
     },
 
     },
     "forum.flightgear.org": {
+
     url: {
        content: {
+
      xpath: 'tbody/tr[1]/td/div/div[1]/b/a/@href',
            selection: getSelectedHtml,
+
      transform: [prepend('https://sourceforge.net')]
            transform: [removeComments(),
+
                        escapePipes(),
+
                        a2wikilink(),
+
                        phpBB_smilies2text(),
+
                        img2link(),
+
                        phpBB_fontstyle2wikistyle(),
+
                        phpBB_code2syntaxhighlight(), // FIXME: could be much better, see below
+
                        phpBB_quote2cquote(),
+
                        escapeEquals(),
+
                        addNewlines()]
+
        },
+
        author: {
+
            xpath: "../p/strong/a/text()"
+
        },
+
        title: {
+
            xpath: "../h3/a/text()"
+
        },
+
        date: {
+
            xpath: "../p/text()[2]",
+
            transform: extract(/ » (.*),/)
+
        },
+
        url: {
+
            xpath: "../p/a/@href",
+
            transform: [extract(/\.(.*)/),
+
                        prepend("http://forum.flightgear.org")]
+
        }
+
    }   
+
};
+
+
var OUTPUT = {
+
    // shows a message box
+
    msgbox: function(msg) {
+
        if (window.prompt ("Copy to clipboard: Ctrl+C, Enter", msg) !== null)
+
            window.getSelection().removeAllRanges(); // deselect all text
+
 
     }
 
     }
};
+
  }, // end of mailing list profile
+
</syntaxhighlight>
//////////////////////
+
// TRANSFORM OPERATORS
+
+
// prepend 'prefix'
+
function prepend(prefix) {
+
    return function(text) {
+
        return prefix+text;
+
    };
+
}
+
+
// extract the first capture group in the regex (i.e. '(xxx)')
+
function extract(regex) {
+
    return function(text) {
+
        return text.match(regex)[1];
+
    };
+
}
+
+
// replaces newlines with "unescaped" <br/>'s
+
function newline2br() {
+
    return function(text) {
+
        return text.replace(/\n/g,"<br/>\n");
+
    };
+
}
+
+
// remove html comments (e.g. '<!-- this is a comment -->')
+
function removeComments() {
+
    return function(html) {
+
        return html.replace(/<!--.*?-->/g,"");
+
    };
+
}
+
+
// converts html <a>...</a>'s to wiki links, internal if possible
+
function a2wikilink() {
+
    return function(html) {
+
        // first tries internal links, firstmost links to images (File:xxx), because
+
        // they need special treatment, or they get displayed
+
        html = html.replace(/<a[^(?:href)]*href="https?:\/\/wiki.flightgear.org\/(File:.*?)".*?>(.*?)<\/a>/g, "[[:$1|$2]]");
+
        html = html.replace(/<a[^(?:href)]*href="https?:\/\/wiki.flightgear.org\/(.*?)".*?>(.*?)<\/a>/g, "[[$1|$2]]");
+
       
+
        // then the others
+
        html = html.replace(/<a[^(?:href)]*href="(.*?)".*?>(.*?)<\/a>/g, "[$1 $2]");
+
        return html;
+
    };
+
}
+
+
// converts embedded html images to external links,
+
// but shows them as little images if they're from the wiki
+
function img2link() {
+
    return function(html) {
+
        html = html.replace(/<img.*?src="https?:\/\/wiki.flightgear.org\/images\/.*?\/([^\/]*?)".*?>/g, "[[File:$1|250px]]");
+
        return html.replace(/<img.*?src="(.*?)".*?>/g, "(see the [$1 linked image])");
+
    };
+
}
+
  
// puts newlines where it makes for more readable wiki "source"
+
Now, we need to review/adapt the profile according to the new archive we'd like to see supported.  
function addNewlines() {
+
    return function(html) {
+
        html = html.replace(/<br\/?>/g,"<br/>\n");
+
        html = html.replace(/(<\/?[uo]l>)/g,"$1\n");
+
        return html.replace(/<\/li>/g,"</li>\n");
+
    }
+
}
+
+
// converts phpBB way of doing font style to the wiki way
+
function phpBB_fontstyle2wikistyle() {
+
    return function(bbhtml) {
+
        bbhtml = bbhtml.replace(/<span style="font-weight: bold">(.*?)<\/span>/g, "'''$1'''");
+
        bbhtml = bbhtml.replace(/<span style="text-decoration: underline">(.*?)<\/span>/g, "<u>$1</u>");
+
        bbhtml = bbhtml.replace(/<span style="font-style: italic">(.*?)<\/span>/g, "''$1''");
+
        bbhtml = bbhtml.replace(/<span class="posthilit">(.*?)<\/span>/g, "$1");
+
        return bbhtml;
+
    };
+
}
+
+
// converts (html) phpBB code blocks to wiki <syntaxhighlight>
+
// FIXME: not actually using the above tag, because the copied html has
+
//        unconverted html entities and br's are not interpreted.
+
//        Solution would be to extract the code, fix it and use the proper tag...
+
function phpBB_code2syntaxhighlight() {
+
    return function(bbhtml) {
+
        return bbhtml.replace(/<dl.*?><dt>.*?<\/dt><dd><code>(.*?)<\/code><\/dd><\/dl>/g, "<tt>\n$1\n</tt>");
+
    };
+
}
+
+
// converts (html) phpBB quotes to simple wiki cquotes (author not cited, it'd get messy anyway)
+
function phpBB_quote2cquote() {
+
    return function(bbhtml) {
+
        return bbhtml.replace(/<blockquote.*?><div>(?:<cite>.*?<\/cite>)?(.*?)<\/div><\/blockquote>/g, "{{cquote|$1}}");
+
    };
+
}
+
  
// converts smilies images from phpBB.
+
for starters, that means:
// Must be used BEFORE img2link(), because it makes a broken link of them
+
* changing the name of the profile, e.g. to read gmane (instead of sourceforge)
function phpBB_smilies2text() {
+
* change the '''url_reg''' field to the gmane URL (this can be a regular expression)
    return function(bbhtml) {
+
 
        return bbhtml.replace(/<img.*?src="\.\/images\/smilies\/icon_.*?\.gif".*?alt="(.*?)".*?>/g,"$1");
+
Next, it makes sense to use an [https://addons.mozilla.org/de/firefox/addon/xpath-checker/ XPath checker], so that we can look up the xpath expression for various HTML elements, and add those to the configuration hash above.
    }
+
 
}
+
For testing purposes, you will probably go to the setup dialog and enable the DEBUG mode, and use your browser's console to see what is going on.
+
 
// escapes pipe characters that can be problematic inside a template.
+
=== Getting involved ===
// Must be used BEFORE html tags were converted, or they get messed up.
+
While having some experience with JavaScript/HTML and jQuery will definitely be useful, JavaScript is close enough to FlightGear scripting ([[Nasal]]), so that people can get involved pretty easily.  
function escapePipes() {
+
 
    return function(text) {
+
Most maintenance work will typically involve reviewing/maintaining a few configuration hashes, that contain meta information for each supported archive (mailing list/forum).
        text = text.replace(/\|\|/g,"{{!!}}");
+
 
        text = text.replace(/\|\-/g,"{{!-}}");
+
Usually, each hash contains a combination of xpath/regex expressions to look up the relevant information, as well as vector of optional transformations that are applied (in order) to convert contents to a different format (e.g. dates).
        return text.replace(/\|/g,"{{!}}");
+
 
    }
+
In addition, there is growing library of utility functions, a handful wrappers for useful stuff, for example:
}
+
 
+
* <code>Host.dbLog(message_string)</code> - log a message to the console if the DEBUG flag is set
// escapes the equal sign that can be problematic inside a template.
+
* <code>Host.download(url, callback, method='GET')</code> - will download the URL and pass the downloaded content to the callback specified
// Must be used AFTER html tags were converted, or they get messed up.
+
* <code>Host.downloadPosting(url, callback)</code> - will download the posting URL and pass the extracted and transformed author/content and date fields in a hash to the callback specified, the URL must be one supported in the CONFIG hash (i.e. forum/sourceforge for now)
function escapeEquals() {
+
* <code>Host.make_doc(string, type="text/html")</code> - will turn a string/blob into a DOM that can be queried
    return function(text) {
+
* <code>Host.eval_xpath(document, xpath_expression, type=XPathResult.STRING_TYPE)</code> - will apply the xpath expression to the document specified, returning the requested type (defaulted to string)  
        return text.replace(/=/g,"{{=}}");
+
* <code>Host.set_persistent(key,value)</code> - stores a key/value pair
    }
+
* <code>Host.get_persistent(key,default_value)</code> - retrieves a value using the key specified, falling back to the default value specified
}
+
 
   
+
=== Self checks (unit testing) ===
+
For regression testing purposes, there's a dedicated "self check" dialog that will be extended over time. For now it will download a few profile/website specific postings using a vector called "tests" and then log the posting's title, author and date to the console.
///////////////////
+
 
// SCRIPT MAIN BODY
+
The next step will be actually showing that information in the dialog itself - there's a separate helper function to accomplish that, so that the corresponding div layer can be updated with the results.
+
 
window.addEventListener('load', register);
+
[[File:Sanity-check-cquotes-dialog.png|thumb|automatically executed sanity checks]]
+
 
function register() {
+
=== Debug mode ===
    // check if we have a matching domain in the CONFIG hash
+
[[File:Instant-cquotes-debug-mode.png|thumb|Instant-Cquotes debug mode]]
    if (CONFIG.hasOwnProperty(window.location.hostname)) {
+
 
        document.onmouseup = instantCquote; // register the callback for "mouse button up" event
+
 
    }
+
=== AJAX (live page editing) ===
}
+
{{WIP}}
   
+
 
// main function
+
* http://wiki.flightgear.org/api.php
function instantCquote() {
+
* http://wiki.flightgear.org/api.php?action=parse&page=Frequently%20asked%20questions&prop=sections
    if(getSelectedText() === "") return;
+
 
+
For now, the setup dialog will try to obtain a login token for the wiki and show a message if successful.
    var profile = CONFIG[window.location.hostname];
+
 
    var parent = getSelectionParent();
+
In addition, the profile/website hash also contains a new wiki entry for the FlightGear wiki, whose '''event_handler''' callback will be invoked once the FG wiki is visited - the console/log will show a greeting, so that is where other code can be added - e.g. to help clean up/rewrite FGCquote-based articles automatically etc.
+
 
    var info = parent ? extractInfo(profile, parent) : extractInfoFallback();
+
There is a vector of "modes", whose members are a hash containing trigger/handler fields, linked to two callbacks - the trigger callback can be used to check some condition, while the handler will be invoked if the trigger returns true.
+
 
    if (info) {
+
This can be used to support an arbitrary number of modes, whose triggers are evaluated during page load - all triggers that return true, will have their handlers invoked, which is how the following snippet works to rewrite/augment wiki edit handles:
        var cquote = createCquote(info);
+
 
        OUTPUT.msgbox(cquote);
+
<syntaxhighlight lang="javascript>
    } else {
+
  'FlightGear.wiki': {
        alert("info == null");
+
    type: 'wiki',
    }
+
     enabled: false,
}
+
    event: 'document.onmouseup', // when to invoke the event handler
+
     event_handler: function () {
function extractInfoFallback() { // XXX untested
+
      console.log('FlightGear wiki handler active (waiting to be populated)');
    var info = {};
+
      // this is where the logic for a wiki mode can be added over time (for now, it's a NOP)
    info.content = getSelectedText();
+
   
    info.url = document.URL;
+
    //for each supported mode, invoke the trigger and call the corresponding handler
    info.title = "from " + window.location.hostname;
+
    [].forEach.call(CONFIG['FlightGear.wiki'].modes, function(mode) {
    info.author = "";
+
      //dbLog("Checking trigger:"+mode.name);
    info.date = "";
+
      if(mode.trigger) {
    return info;
+
        mode.handler();
}
+
      }
   
+
    });
function extractInfo(profile, parent) {
+
     
     var info = {};
+
     }, // the event handler to be invoked
+
    url_reg: '^(http|https)://wiki.flightgear.org', // ignore for now: not currently used by the wiki mode
     for (var field in profile) {
+
      
        if (!profile.hasOwnProperty(field)) continue;  
+
    modes: [
+
      { name:'process-editSections',
            // this part gets the data, both for field and content (i.e. text, html)
+
        trigger: function() {return true;}, // match URL regex - return true for always match
            var fieldInfo = extractFieldInfo(profile, parent, field);
+
     
+
         // the code implementing the mode
            if (debug) alert("pre trans:\n" + field + " = " + fieldInfo);
+
         handler: function() {
+
               
            var transform = profile[field].transform;
+
    var editSections = document.getElementsByClassName('mw-editsection');
            if(transform) fieldInfo = applyTransformations(fieldInfo, transform);
+
     console.log('FlightGear wiki article, number of edit sections: '+editSections.length);
+
 
            if (debug) alert("post trans:\n" + field + " = " + fieldInfo);
+
    // for now, just rewrite edit sections and add a note to them
+
 
            info[field] = fieldInfo;
+
    [].forEach.call(editSections, function (sec) {
+
      sec.appendChild(
     }
+
        document.createTextNode(' (instant-cquotes is lurking) ')
+
      );
     return info;
+
    }); //forEach section
}
+
         } // handler
+
     
function extractFieldInfo(profile, parent, field) {
+
     
    if (field != "content") {
+
      } // process-editSections
         var xpath = profile[field].xpath;
+
      // TODO: add other wiki modes below
         var parentXPath = getXPathTo(parent);
+
     
        var fullPath = parentXPath + "/" + xpath;
+
     ] // modes
        return document.evaluate(fullPath, document, null, XPathResult.STRING_TYPE, null).stringValue;
+
      
     } else {
+
   }, // end of wiki profile
        return profile[field].selection(); // here we extract the contents of the selection
+
</syntaxhighlight>
    }
+
 
}
+
 
+
See [[User:Red Leader/Sandbox/AJAX test]]
function applyTransformations(fieldInfo, trans) {
+
 
    if(typeof trans === "function") {
+
* https://www.mediawiki.org/wiki/API:Changing_wiki_content
        return trans(fieldInfo);
+
* https://www.mediawiki.org/wiki/Extension:VisualEditor
    } else if (Array.isArray(trans)) {
+
* https://en.wikipedia.org/wiki/Wikipedia:Creating_a_bot
        for (var i = 0; i < trans.length; i++) {
+
 
            if (debug) alert("pre trans in array:\n = " + fieldInfo);
+
=== Mobile edition ===
            fieldInfo = trans[i](fieldInfo);
+
 
            if (debug) alert("post trans in array:\n = " + fieldInfo);
+
=== Issues/limitations ===
         }
+
==== Bugs ====
        return fieldInfo;
+
* It's eating characters, apparently related to regex/xpath handling - e.g. words like "analyzing" are turned into "analying" [http://forum.flightgear.org/viewtopic.php?f=6&t=28378&p=270735&hilit=analyzing#p270735] [http://wiki.flightgear.org/index.php?title=FlightGear_Newsletter_May_2016&curid=15768&diff=99527&oldid=99258]
    }
+
 
}
+
==== Non working URLs ====
+
* forum2wiki img conversion does not work for all embedded images, e.g. see https://forum.flightgear.org/viewtopic.php?f=4&t=25747&p=285956#p285956
function createCquote(info) {
+
* image matching/extraction: http://forum.flightgear.org/viewtopic.php?p=276221#p276221
    //TODO: add template string to profile
+
* http://sourceforge.net/p/flightgear/mailman/message/34754961/
    var template = "{{FGCquote" + "\n" +
+
* http://forum.flightgear.org/viewtopic.php?f=18&t=27054&start=90#p273972 → selecting from “As promised, two sample installation sessions on Linux” to “That's it.” towards the end of the message causes Iceweasel (Firefox) 44.0.2 to display a dialog box reading “A script on this page may be busy, or it may have stopped responding. You can stop the script now, open the script in the debugger, or let the script continue.” The line below reads “Script: chrome://greasemonkey-modules/...quotes/instant_cquotes.user.js:544”. Choosing ''Continue'' doesn't help: the same message reappears a few seconds afterwards.
        "  |" + info.content + "\n" +
+
 
        "  |{{cite web |url=" + info.url + "\n" +
+
=== Feature requests & ideas ===
        "     |title=" + nowiki(info.title) + "\n" +
+
* resolve footers [https://sourceforge.net/p/flightgear/mailman/message/36059822/]: "I believe that because of [x] ... [x]: " -> "I believe that because of (x)"
        "     |author=" + nowiki(info.author) + "\n" +
+
* make transformations customizable/editable via the UI (think git links to repo url template)
        "    |date=" + nowiki(info.date) + "\n" +
+
* split into separate files [https://stackoverflow.com/questions/8695459/is-it-possible-to-split-greasemonkey-user-scripts-into-multiple-files][]
        "   }}" + "\n" +
+
* replace temporary references:
        "}}";
+
** yesterday/today/tomorrow
    return template;
+
** last/this/next week, month/year/release
}
+
* Article wizard, optionally append References section with Appendix
+
* Template integration to turn templates into interactive wizards to create/add to pages, for example:
function createForumQuote(info) {
+
** [[Template:Canvas Snippet]]
    //TODO: add template string to profile
+
** [[Template:Nasal doc]]
    // nowiki(info.date)
+
* try to recognize list items [https://sourceforge.net/p/flightgear/mailman/message/35095319/] (heuristics: look for colon/asterisk, dashes and CR/LF)
    var template = "[url='"+info.url+"']"+info.title+"("+nowiki(info.date)+")[/url][quote='"+nowiki(info.author)+"']" + nowiki(info.content) + "\n" +
+
* should add [[Template:News]] to the article dropdown for announcements [http://wiki.flightgear.org/index.php?title=Template:News&oldid=98266]
        "[/quote]";
+
* add a mode that will download screenshots from forum postings and automatically upload/categorize them, see [[Birds]]
    return template;
+
* split the article dropdown into sections, and also populate it with the user's watchlist [https://www.mediawiki.org/wiki/API:Watchlist] {{Progressbar|60}}
}
+
* mailing list templates, listed at [http://wiki.flightgear.org/Template_talk:Project_infrastructure#Related_mailing_list_templates]
+
* expose the cquote/ref markup via the UI so that it can be edited/customized and treated like a template {{Done}} (0.36+)
+
* identify common/repeated links and automatically create [[Template:Project infrastructure|link/infrastructure templates]] and use those (should be straightforward using the AJAX mode) [http://wiki.flightgear.org/index.php?title=Mailing_lists&curid=2038&diff=97876&oldid=85252]
function nowiki(text) {
+
** use simple naming heuristic: Author_title_date
    return "<nowiki>" + text + "</nowiki>";
+
* add a devel/maintainer mode where it will return the xpath for a selection [http://stackoverflow.com/questions/361130/get-selected-text-and-selected-nodes-on-a-page] [http://stackoverflow.com/questions/12485334/get-surrounding-dom-node-of-selection]
}
+
* move openlink,dblog helpers to Environment hash {{Done}}
+
* identify CLI arguments like --aircraft=c172p and wrap them in between code tags <code>--aircraft=c172p</code> [https://sourceforge.net/p/flightgear/mailman/message/35063277/] (note that we can simply download [https://sourceforge.net/p/flightgear/fgdata/ci/next/tree/options.xml options.xml] via openlink() and use that, which is kinda of neat...) {{Progressbar|60}} (see downloadOptionsXML() in the code)
////////////////////
+
* introduce "layouts" (templates) for different purposes: newsletter, changelog, wiki article, [[The Manual]] (LaTex)  ? {{Progressbar|40}}
// UTILITY FUNCTIONS
+
* use wikipedia template if possible [https://sourceforge.net/p/flightgear/mailman/message/35057670/]
+
* the new '''tests''' vector could also contain vectors for tests to test the extract/transform* utilities, see Environment.APITests {{Progressbar|50}}
// IE8 compat. function
+
* move environment specific APIs (browser, script host etc) into some kind of Environment hash to encapsulate things (Red Leader was working on a pure Chrome-only version at some point IIRC) {{Progressbar|80}}
function getSelectionParent() {
+
* encode script settings in created markup, for future processing/updating of quotes
    if(document.getSelection) { // for modern, standard compliant browsers
+
* look up <code>[x]</code> references and replace with the corresponding link (titled) [https://sourceforge.net/p/flightgear/mailman/message/35055331/] [https://sourceforge.net/p/flightgear/mailman/message/35062598/]
        // anchorNode is the textnode, parentNode is the parent html element
+
** convert footnotes into Abbr templates [http://article.gmane.org/gmane.games.flightgear.devel/78971]
        return document.getSelection().anchorNode.parentNode;
+
* support named refs for combining identical refs [http://wiki.flightgear.org/index.php?title=FlightGear_Qt_launcher&curid=13693&diff=97562&oldid=97551]
+
* adopt {{tl|forum link}}
    } else if (document.selection) { // for IE <= v8 - XXX: not tested!
+
* implement a less obnoxious quoting mode, without quotes, where only the ref part would be added, e.g. see the example at [[Graphics Card Profiles]] (it's still 99% quotes, but much less annoying) {{Done}}
        return document.selection.createRange().parentElement();
+
* attachment support: identify attachments and link to them: [https://sourceforge.net/p/flightgear/mailman/message/11683451/] [https://sourceforge.net/p/flightgear/mailman/message/23906620/] [https://sourceforge.net/p/flightgear/mailman/message/35059842/] [https://sourceforge.net/p/flightgear/mailman/message/35067087/]
    }
+
* bulletin points: if there is a colon (:) followed by at least two dashes (-), split up everything after the colon to turn each dash into an asterisk (wiki markup for bulletin points), followed by a newline [http://sourceforge.net/p/flightgear/mailman/message/34760165/] 
}
+
* generic URL/template matching, e.g. for for sourceforge commit IDs
+
* make filters/conversions configurable via checkboxes (nowiki, wrap in alert/note boxes)
// IE8 compat. function
+
* make syntax highlighting configurable (language, mode) ?
function getSelectedText() {
+
* consider using something like the Roles template to turn contributor names into tooltips where contributor roles are shown (core dev, fgdata committer etc)
    if(document.getSelection) { // for modern, standard compliant browsers
+
* introduce support for tag clouds to help categorize/classify related quotes
        return document.getSelection().toString();
+
* consider making transformations optional/configurable using check boxes in the jQuery dialog
+
* add new input method, for quotes that need to be updated/converted (added script version specifically for this purpose), should also add extraction/processing date
    } else if (document.selection) { // for IE <= v8 - XXX: not tested!
+
* investigate why not all mailing list archives/postings are supported correctly: http://sourceforge.net/p/flightgear/mailman/message/8090479/ (problem traced to <code>getPostID()</code>)
        return document.selection.createRange().text;
+
* explore having a ref-only mode without using cquotes, i.e. just copy/paste quotes with proper ref tags and a references section, to rewrite the whole thing (possibly with templates for different purposes, e.g. newsletter/changelog) {{Done}}
    }
+
* should use {{tl|forum link}} ([[:Category:link templates]])
}
+
* should be updated to use the new repo/flightgear file templates created by Johan & RedLeader {{Not done}}
+
* resolve links to forum threads to look up the title for the topic/posting, so that the posting's title can be used for those links, instead of just the URL - we can probably do that by making an AJAX call to open URLs asynchronously and extract the title for the thread/posting to come up with something like <code>[http://forum.flightgear.org/viewtopic.php?f=4&t=24421 A call to developers-Lockheed -L188 Electra]</code> ([http://forum.flightgear.org/viewtopic.php?f=4&t=26562&p=247325&hilit=links#p247347]) {{Progressbar|50}}
// IE8 compat. function (copied from http://stackoverflow.com/a/6668159 )
+
* convert quoted bug tracker URLs to use the issue template on the wiki {{not done}}
function getSelectedHtml() {
+
* do regex/xpath validation, and display any errors (e.g. template/theme changes on the forum would currently break the script) {{Progressbar|60}}
    var html = "";
+
* increased focus on supporting different output formats, maybe using a simple [http://www.jquery-steps.com/ jQuery based wizard] (wiki, forum, newsletter, changelog) {{Not done}}
    if (typeof window.getSelection != "undefined") {
+
* token matching for keywords/acronyms to link to the corresponding wiki articles (e.g. Nasal, Canvas, FG_ROOT, FG_HOME etc) {{Not done}}
        var sel = window.getSelection();
+
* Add support for [http://sourceforge.net/p/flightgear/codetickets/ tickets], merge requests comments and [http://www.fguk.eu/index.php/forum/index FGUK forum]. (also see [[FlightGear wiki talk:Instant-Cquotes#more sources]])
        if (sel.rangeCount) {
+
* GET-encoded SID arguments should be stripped from forum URLs. {{Not done}}
            var container = document.createElement("div");
+
* Links to repositories should be converted to use wiki templates. {{Not done}}
            for (var i = 0, len = sel.rangeCount; i < len; ++i) {
+
* The {{Abbr|regexes|regular expressions}} used may fail if the HTML DOM of the source changes (e.g., phpBB/theme update)
                container.appendChild(sel.getRangeAt(i).cloneContents());
+
** Show a warning when that's the case. {{Progressbar|70}} (see the self-check dialog)
            }
+
** Try multiple regexes in order. {{Progressbar|30}} (see the self-check dialog)
            html = container.innerHTML;
+
* Use the script to update previously created Cquotes automatically
        }
+
** Instead of using the <code>getSelection()</code> helper, we could register a match for <tt>wiki.flightgear.org</tt> with <code>action=edit</code> set, so that we can directly process all text of an edited page, using AJAX calls to open the URL in the background. {{Progressbar|40}} (see the self-check dialog, available via the greasemonkey menu)
    } else if (typeof document.selection != "undefined") {
+
** See [https://www.mediawiki.org/wiki/API:Edit#Editing_via_Ajax MW:API:Edit § Editing via Ajax]
        if (document.selection.type == "Text") {
+
 
            html = document.selection.createRange().htmlText;
+
== The script ==
        }
+
<gallery mode=packed widths=230px heights=230px>
    }
+
Instant-cquotes-revamped.png|Instant cquotes: Revamped user interface exposes script internals to make the script better configurable at runtime
    return html;
+
Instant-cquotes-template-editor.png|Instant cquotes now features a simple built-in template editor with support for variable substitution, so that wiki templates can be more easily customized
}
+
Instant-cquotes-with-wikimedia-API-integration.png|Screenshot showing instant-cquotes with wikimedia API integration to fetch articles/sections and populate dropdown menus accordingly
+
Instant-cquotes-on-steroids-with-watchlist-support.png|Screenshot showing the JQuery-mode of the instant cquotes script with built-in support for fetching a user's wiki/watchlist to edit/update articles in a semi-automated fashion
// copied from http://stackoverflow.com/a/2631931
+
</gallery>
function getXPathTo(element) {
+
 
    if (element.id !== '')
+
{{Note|
        return 'id("' + element.id + '")';
+
The userscript times are over meanwhile, the script has been ported to use the web extension framework - please don't update the user script. thanks !
    if (element === document.body)
+
}}
        return element.tagName;
+
+
    var ix= 0;
+
    var siblings = element.parentNode.childNodes;
+
    for (var i= 0; i<siblings.length; i++) {
+
        var sibling = siblings[i];
+
        if (sibling === element)
+
            return getXPathTo(element.parentNode) + '/' + element.tagName + '[' + (ix+1) + ']';
+
        if (sibling.nodeType === 1 && sibling.tagName === element.tagName)
+
            ix++;
+
    }
+
}</syntaxhighlight>
+
  
[[Category:Wiki maintenance]]
+
{{Appendix}}

Latest revision as of 04:15, 5 June 2019

Quotes-logo-200x200.png
Instant-Cquotes user script in Firefox
Screenshot showing the web-extension port of the InstantRefs user-script in FireFox.
Instant-Cquotes screenshot prototyping runtime format selection

The Instant-Refs (renamed in 11/2016, originally, Instant-cquotes) script is a browser addon (as of late 2017, a so called web-extension) implemented in JavaScript that started out as a user script in order to convert excerpts (created via copy&paste) from FlightGear forum or mailing list postings into MediaWiki markup/quotes to be used on the FlightGear wiki.

It is being developed and maintained by a group of volunteers involved in maintaining the wiki and in trying to provide more up-to-date information to end-users who may not be as involved in the various FlightGear-related communication channels.

Background and motivation

FlightGear's development is, at best, "self-coordinated", meaning that contributors discuss ideas and make proposals to contribute in a certain fashion and then team up to implement certain features and building blocks, often just temporarily.

Unfortunately, due to a lack of development manpower, many ideas are not implemented immediately; it is, thus, important to know their pros and cons, as well as who originally proposed them and/or might help with their implementation, even after a long time, preferably with links to the original discussions so that new contributors can decide whether to get involved in some effort or not. The project documentation, however, is in great need of improvement[1] and usually significantly lacking behind:[2] many core developers update it seldomly, if not anymore, as it takes time to write it, as well as an understanding of the inner workings of a complex system such as FlightGear. Furthermore, this task might not be attractive due to its short term impact on the project,[3] and it is overwhelming to think about creating documentation that would address the needs of many different kinds of contributors with different backgrounds, experience levels and goals.[2]

Forum and mailing lists discussions have therefore become the only up-to-date (albeit difficult to filter)[4] source of information about recent development progress; this makes it tricky to know what is going on, what needs fixing, what were the decisions taken by the developers.[5]

The aim of the Instant-Refs script is to help wiki editors to copy relevant excerpts from such sources, formatting them as proper quotations, and help bootstrap new articles collecting them until a dedicated rewrite is made. It can also be used to reuse announcements to update the changelogs, newsletter or the Release plan/Lessons learned page.

After being away from the wiki system for some time it becomes more of an effort to re-learn how to start a new file and figure out how to format it , with what headings, etc. Sometimes it is almost too much to do the original write up of the original post and all the work of that layout and effort to have to also duplicate it in a wiki article. If I was a little more current and affluent with the wiki editor, I might not be so hesitant to create the work there instead of the forum.[6]


In a few cases, such collections of quotes helped not only create bootstrap new articles, but even actual features, too.

In other cases, quotes have been used to update documentation of features whose maintainers may not be actively involved in FlightGear (e.g. Rembrandt), to help document discussions that are taking place in the meantime, and provide some background information for people interested in the corresponding feature.

Configuration

GreaseMonkey menu shown in FireFox with instanct cquotes menu items
The configuration dialog for the Instant-Cquotes script

As of version 0.30, a dedicated configuration dialog is in the process of being added, so that certain script features can be dynamically configured, without having to edit the script. For now, this is merely a placeholder that provides a checkbox to easily enable/disable the debug mode. In the future, we are hoping to also expose other features this way.

Usage

Instant-Cquotes script, with updates contributed by Red Leader
Screenshot showing Instant-Cquotes 0.30 at work
  1. Go to some mailing list archive URL, for example [1] or any forum message, such as Re: Get objects to show up on Map/Radar topic on the forum This is a link to the FlightGear forum..
  2. Select the relevant portion of text.
  3. When you release the mouse button, a box will appear containing the converted text (for now, mainly properly-referenced quotes for the wiki).
  4. The text will be automatically selected and copied to the clipboard.
  5. Paste the text into the desired wiki page.

For example, by selecting part of the forum post in the link above you can get the following quotation:

Cquote1.png The upcoming FlightGear version (3.2) will contain a canvas-based map dialog, including a modular "plugin" system for creating custom map layers and charts with roughly ~50 lines of code, most of it boilerplate.

This is entirely XML/Nasal based (scripted) - symbols can be pretty much anything, raster or vector images (png or svg), but even animated. Styling can be customied, too. For more info, I suggest to check out: MapStructure#Porting the map dialog

MapStructureDialog.png
— Hooray (Jun 14th, 2014). Re: Get objects to show up on Map/Radar.
(powered by Instant-Cquotes)
Cquote2.png

On quoting

1rightarrow.png See FlightGear wiki:Quoting Guidelines for the main article about this subject.

Using the Instant-Cquotes script is a good way to bootstrap and write some preliminary notes; however, while quotes might be useful to understand how undocumented subsystems and features work and are definitely better than nothing, they are not meant to replace proper, structured and well-written wiki articles.[7]

One way to convert pages bootstrapped using quotes is to extract relevant information from them and keep citations only as references; in case important details are missing, they can be asked for on the mailing lists (on the forum, the chance to get a complete answer might be lower).[8] Another option might be moving the quotes to the Talk page for each entry, which would preserve the sources without clogging up the articles.[9]

As a matter of fact, the whole paragraph above was assembled using this approach; to see for yourself, look up the references at the end of this page. For another example, see TerraSync#News.

Development

Note  A Chrome/Chromium-specific extension that will not need Tampermonkey installed is under development.

Resources

Adding sources

Adding a new source is pretty straightforward if you understand how xpath and regexes work - basically, you only need an archive (e.g. gmane), and then determine the xpath of each relevant field, as well as the regular expression to process the extracted fields (optional).

The basic steps are these:

  1. open the user script in an editor
  2. navigate to the meta header of the user script
  3. add a new URL to the top of the script, e.g. by copying/adapting an existing line like this:
// @match       https://sourceforge.net/p/flightgear/mailman/*

Once copied, add the new URL:

// @match       http://thread.gmane.org/gmane.games.flightgear.devel/*

Next, you need to navigate to the configuration hash to add a new website to it.

Again, it makes sense to simply take an existing configuration hash and adapt it as needed (ignore/omit the tests vector for now by keeping it empty):

Caution  the following example may meanwhile be outdated, so be sure to look at the actual code instead
  'Sourceforge Mailing list': {
    enabled: true,
    type: 'archive',
    event: 'document.onmouseup', // when to invoke the event handler
    event_handler: instantCquote, // the event handler to be invoked
    url_reg: '^(http|https)://sourceforge.net/p/flightgear/mailman/.*/',
    content: {
      xpath: 'tbody/tr[2]/td/pre/text()',
      selection: getSelectedText,
      idStyle: /msg[0-9]{8}/,
      parentTag: [
        'tagName',
        'PRE'
      ],
    transform: [] // vector with transformation callbacks
    }, // content recipe
    // vector with tests to be executed for sanity checks (unit testing)
    tests: [
    ], // end of vector with self-tests
    // regex/xpath and transformations for extracting various required fields
    author: {
      xpath: 'tbody/tr[1]/td/div/small/text()',
      transform: [extract(/From: (.*) <.*@.*>/)]
    },
    title: {
      xpath: 'tbody/tr[1]/td/div/div[1]/b/a/text()'
      transform: []
    },
    date: {
      xpath: 'tbody/tr[1]/td/div/small/text()',
      transform: [extract(/- (.*-.*-.*) /)]
    },
    url: {
      xpath: 'tbody/tr[1]/td/div/div[1]/b/a/@href',
      transform: [prepend('https://sourceforge.net')]
    }
  }, // end of mailing list profile

Now, we need to review/adapt the profile according to the new archive we'd like to see supported.

for starters, that means:

  • changing the name of the profile, e.g. to read gmane (instead of sourceforge)
  • change the url_reg field to the gmane URL (this can be a regular expression)

Next, it makes sense to use an XPath checker, so that we can look up the xpath expression for various HTML elements, and add those to the configuration hash above.

For testing purposes, you will probably go to the setup dialog and enable the DEBUG mode, and use your browser's console to see what is going on.

Getting involved

While having some experience with JavaScript/HTML and jQuery will definitely be useful, JavaScript is close enough to FlightGear scripting (Nasal), so that people can get involved pretty easily.

Most maintenance work will typically involve reviewing/maintaining a few configuration hashes, that contain meta information for each supported archive (mailing list/forum).

Usually, each hash contains a combination of xpath/regex expressions to look up the relevant information, as well as vector of optional transformations that are applied (in order) to convert contents to a different format (e.g. dates).

In addition, there is growing library of utility functions, a handful wrappers for useful stuff, for example:

  • Host.dbLog(message_string) - log a message to the console if the DEBUG flag is set
  • Host.download(url, callback, method='GET') - will download the URL and pass the downloaded content to the callback specified
  • Host.downloadPosting(url, callback) - will download the posting URL and pass the extracted and transformed author/content and date fields in a hash to the callback specified, the URL must be one supported in the CONFIG hash (i.e. forum/sourceforge for now)
  • Host.make_doc(string, type="text/html") - will turn a string/blob into a DOM that can be queried
  • Host.eval_xpath(document, xpath_expression, type=XPathResult.STRING_TYPE) - will apply the xpath expression to the document specified, returning the requested type (defaulted to string)
  • Host.set_persistent(key,value) - stores a key/value pair
  • Host.get_persistent(key,default_value) - retrieves a value using the key specified, falling back to the default value specified

Self checks (unit testing)

For regression testing purposes, there's a dedicated "self check" dialog that will be extended over time. For now it will download a few profile/website specific postings using a vector called "tests" and then log the posting's title, author and date to the console.

The next step will be actually showing that information in the dialog itself - there's a separate helper function to accomplish that, so that the corresponding div layer can be updated with the results.

automatically executed sanity checks

Debug mode

Instant-Cquotes debug mode


AJAX (live page editing)

WIP.png Work in progress
This article or section will be worked on in the upcoming hours or days.
See history for the latest developments.

For now, the setup dialog will try to obtain a login token for the wiki and show a message if successful.

In addition, the profile/website hash also contains a new wiki entry for the FlightGear wiki, whose event_handler callback will be invoked once the FG wiki is visited - the console/log will show a greeting, so that is where other code can be added - e.g. to help clean up/rewrite FGCquote-based articles automatically etc.

There is a vector of "modes", whose members are a hash containing trigger/handler fields, linked to two callbacks - the trigger callback can be used to check some condition, while the handler will be invoked if the trigger returns true.

This can be used to support an arbitrary number of modes, whose triggers are evaluated during page load - all triggers that return true, will have their handlers invoked, which is how the following snippet works to rewrite/augment wiki edit handles:

 'FlightGear.wiki': {
    type: 'wiki',
    enabled: false,
    event: 'document.onmouseup', // when to invoke the event handler
    event_handler: function () {
      console.log('FlightGear wiki handler active (waiting to be populated)');
      // this is where the logic for a wiki mode can be added over time (for now, it's a NOP)
    
    //for each supported mode, invoke the trigger and call the corresponding handler
    [].forEach.call(CONFIG['FlightGear.wiki'].modes, function(mode) {
      //dbLog("Checking trigger:"+mode.name);
      if(mode.trigger) {
        mode.handler();
      }
    });
      
    }, // the event handler to be invoked
    url_reg: '^(http|https)://wiki.flightgear.org', // ignore for now: not currently used by the wiki mode
    
    modes: [
      { name:'process-editSections',
        trigger: function() {return true;}, // match URL regex - return true for always match
       
        // the code implementing the mode
        handler: function() {
                
    var editSections = document.getElementsByClassName('mw-editsection');
    console.log('FlightGear wiki article, number of edit sections: '+editSections.length);
   
    // for now, just rewrite edit sections and add a note to them
   
     [].forEach.call(editSections, function (sec) {
       sec.appendChild(
         document.createTextNode(' (instant-cquotes is lurking) ')
       );
     }); //forEach section
        } // handler
       
       
      } // process-editSections
      // TODO: add other wiki modes below 
      
    ] // modes
    
  }, // end of wiki profile


See User:Red Leader/Sandbox/AJAX test

Mobile edition

Issues/limitations

Bugs

  • It's eating characters, apparently related to regex/xpath handling - e.g. words like "analyzing" are turned into "analying" [2] [3]

Non working URLs

Feature requests & ideas

  • resolve footers [4]: "I believe that because of [x] ... [x]: " -> "I believe that because of (x)"
  • make transformations customizable/editable via the UI (think git links to repo url template)
  • split into separate files [5][]
  • replace temporary references:
    • yesterday/today/tomorrow
    • last/this/next week, month/year/release
  • Article wizard, optionally append References section with Appendix
  • Template integration to turn templates into interactive wizards to create/add to pages, for example:
  • try to recognize list items [6] (heuristics: look for colon/asterisk, dashes and CR/LF)
  • should add Template:News to the article dropdown for announcements [7]
  • add a mode that will download screenshots from forum postings and automatically upload/categorize them, see Birds
  • split the article dropdown into sections, and also populate it with the user's watchlist [8] 60}% completed
  • mailing list templates, listed at [9]
  • expose the cquote/ref markup via the UI so that it can be edited/customized and treated like a template Done Done (0.36+)
  • identify common/repeated links and automatically create link/infrastructure templates and use those (should be straightforward using the AJAX mode) [10]
    • use simple naming heuristic: Author_title_date
  • add a devel/maintainer mode where it will return the xpath for a selection [11] [12]
  • move openlink,dblog helpers to Environment hash Done Done
  • identify CLI arguments like --aircraft=c172p and wrap them in between code tags --aircraft=c172p [13] (note that we can simply download options.xml via openlink() and use that, which is kinda of neat...) 60}% completed (see downloadOptionsXML() in the code)
  • introduce "layouts" (templates) for different purposes: newsletter, changelog, wiki article, The Manual (LaTex)  ? 40}% completed
  • use wikipedia template if possible [14]
  • the new tests vector could also contain vectors for tests to test the extract/transform* utilities, see Environment.APITests 50}% completed
  • move environment specific APIs (browser, script host etc) into some kind of Environment hash to encapsulate things (Red Leader was working on a pure Chrome-only version at some point IIRC) 80}% completed
  • encode script settings in created markup, for future processing/updating of quotes
  • look up [x] references and replace with the corresponding link (titled) [15] [16]
    • convert footnotes into Abbr templates [17]
  • support named refs for combining identical refs [18]
  • adopt {{forum link}}
  • implement a less obnoxious quoting mode, without quotes, where only the ref part would be added, e.g. see the example at Graphics Card Profiles (it's still 99% quotes, but much less annoying) Done Done
  • attachment support: identify attachments and link to them: [19] [20] [21] [22]
  • bulletin points: if there is a colon (:) followed by at least two dashes (-), split up everything after the colon to turn each dash into an asterisk (wiki markup for bulletin points), followed by a newline [23]
  • generic URL/template matching, e.g. for for sourceforge commit IDs
  • make filters/conversions configurable via checkboxes (nowiki, wrap in alert/note boxes)
  • make syntax highlighting configurable (language, mode) ?
  • consider using something like the Roles template to turn contributor names into tooltips where contributor roles are shown (core dev, fgdata committer etc)
  • introduce support for tag clouds to help categorize/classify related quotes
  • consider making transformations optional/configurable using check boxes in the jQuery dialog
  • add new input method, for quotes that need to be updated/converted (added script version specifically for this purpose), should also add extraction/processing date
  • investigate why not all mailing list archives/postings are supported correctly: http://sourceforge.net/p/flightgear/mailman/message/8090479/ (problem traced to getPostID())
  • explore having a ref-only mode without using cquotes, i.e. just copy/paste quotes with proper ref tags and a references section, to rewrite the whole thing (possibly with templates for different purposes, e.g. newsletter/changelog) Done Done
  • should use {{forum link}} (Category:link templates)
  • should be updated to use the new repo/flightgear file templates created by Johan & RedLeader Not done Not done
  • resolve links to forum threads to look up the title for the topic/posting, so that the posting's title can be used for those links, instead of just the URL - we can probably do that by making an AJAX call to open URLs asynchronously and extract the title for the thread/posting to come up with something like A call to developers-Lockheed -L188 Electra ([24]) 50}% completed
  • convert quoted bug tracker URLs to use the issue template on the wiki Not done Not done
  • do regex/xpath validation, and display any errors (e.g. template/theme changes on the forum would currently break the script) 60}% completed
  • increased focus on supporting different output formats, maybe using a simple jQuery based wizard (wiki, forum, newsletter, changelog) Not done Not done
  • token matching for keywords/acronyms to link to the corresponding wiki articles (e.g. Nasal, Canvas, FG_ROOT, FG_HOME etc) Not done Not done
  • Add support for tickets, merge requests comments and FGUK forum. (also see FlightGear wiki talk:Instant-Cquotes#more sources)
  • GET-encoded SID arguments should be stripped from forum URLs. Not done Not done
  • Links to repositories should be converted to use wiki templates. Not done Not done
  • The regexes used may fail if the HTML DOM of the source changes (e.g., phpBB/theme update)
    • Show a warning when that's the case. 70}% completed (see the self-check dialog)
    • Try multiple regexes in order. 30}% completed (see the self-check dialog)
  • Use the script to update previously created Cquotes automatically
    • Instead of using the getSelection() helper, we could register a match for wiki.flightgear.org with action=edit set, so that we can directly process all text of an edited page, using AJAX calls to open the URL in the background. 40}% completed (see the self-check dialog, available via the greasemonkey menu)
    • See MW:API:Edit § Editing via Ajax

The script

Note

The userscript times are over meanwhile, the script has been ported to use the web extension framework - please don't update the user script. thanks !

References
  1. John Denker (Jul 16th, 2007). [Flightgear-devel] development process (was: chaos...).
  2. 2.0 2.1 Curtis Olson (Jul 27th, 2011). Re: [Flightgear-devel] The state of things in Flight Gear.
  3. Hal V. Engel (Jul 27th, 2011). Re: [Flightgear-devel] The state of things in Flight Gear.
  4. Thorsten (Mar 21st, 2016). Re: quoting on the wiki.
  5. James Turner (Mon, 28 Jul 2008 10:06:05 -0700). [Flightgear-devel] Project tracking.
  6. wlbragg  (May 9th, 2016).  Re: Breaking down NLCD raster image .
  7. bugman (Apr 17th, 2016). Re: What is the QT launcher?.
  8. James Turner (Mar 21st, 2016). Re: [Flightgear-devel] Wiki Quotes.
  9. Stuart Buchanan (Mar 19th, 2016). Re: [Flightgear-devel] Wiki Quotes.