Talk:Switching default texture format to DDS

devel list quotes: yay or nay ?

been in touch with Thorsten about the quotes I started adding before anybody got involved in the article, and working towards summarizing the main points to see if things can be cleaned up a bit. However, this might be one of the few cases, where having a bunch of quotes could actually be a good thing, opinions ? --Hooray (talk) 12:44, 4 September 2014 (UTC)

In this case I think it is a very good idea to use quotes here as many people can be "passionately opinionated" about things like these. It might be very important for the reader to have a good idea of who said what. I even took a look at the devel list at sourceforge, but I have to say that the quotes are easier to follow. While I am at it, thank you for putting the quotes in different section depending on content.

—Johan G (Talk | contribs) 01:06, 5 September 2014 (UTC)

ok, I used bigstones' Instant-Cquotes script to replace those quotes with proper cquotes now - even though Gijs is right, the formatting is still not quite right, I guess we need to update the script eventually to remove redundant stuff like all those <br/> tags. Also, some quotes should probably be re-arranged and shuffled around, like you say: having proper sections is a good idea and helps navigating all those quotes, so we could add a few more sections and re-arrange things accordingly. And like you said, the legalities were already raised 2.5 years ago on the devel list, but for some reason people tend to forget such things ...--Hooray (talk) 01:48, 5 September 2014 (UTC)

Using patented algorithms

I assume there are plugins for example for GIMP out there and perhaps also specialized tools for creating the textures, but something I have not got a clear answer to when searching the web briefly is how one would legally create the compressed textures if the compression algorithms are patented. Unfortunately, at this point it does not seem possible.

This is a bit of a pickle for me. While we have been firm about using non-free resources (though unfortunately some seem to not be). I would say that using patented algorithms without the permission of the patent holder (or his representatives) would be just as bad as doing complete airport layouts using aerial footage form Google maps, or using sound captured from non-free movies in aircraft (or using FlightGear with a stolen joystick...). Note the distinct difference between being published and being free for use.

Browsing a bit more it seems that while there is support for these formats as OpenGL extensions, they still have to be licensed in order to use them. While I am not a lawyer and me be a bit confused, have a look at the section "IP Status" in http://www.opengl.org/registry/specs/EXT/texture_compression_s3tc.txt and at https://www.khronos.org/files/ip-disclosures/opengl/S3_IP_Cert_NOV_2011_clean.pdf (The Khronos Group is the not for profit industry consortium that have written and published latest versions OpenGL specification).

I have also seen this note about using these compression algorithms with Mesa:

S3TC support has been partially implemented in Mesa. We cannot integrate and enable S3TC code by default due to the patents on the algorithm. ... Please be aware of the patent law in your own country with respect to using this code, as it is illegal in many places without a proper license from the patent holder. — freedesktop.org – dri/S3TC. Published by freedesktop.org. Retrieved 5 September 2014.

Based on this I would say that we do not make DDS textures default. In fact they should probably not have been added in the first place. :-(

Sorry if I am a party-pooper...

—Johan G (Talk | contribs) 01:06, 5 September 2014 (UTC)

I am not going to discuss the patent stuff. Please search the mesa-dev archives for the discussion and see there why they think that the nvidia tools and other stuff out there cannot be used. Really - it is not that the code for that is too dificult or unavailable. I am not a lawyer and I cannot change this world - even if I would like to in this regard. I agree that techically for drivers/gpus supporting these compression formats it would be best to use these precompressed files. — Mathias Fröhlich (2012-01-15). Re: [Flightgear-devel] DDS texures (Was: Improving random trees &. (powered by Instant-Cquotes)

Proposal for a change input format "on the fly" in the texture format

At the end of it all, I have always found problems when I used proprietary formats for my programming work. Consider the case of Java, a language with which development every day, when Sun was a lot of promises on possibility to open Java development, but when became property of Oracle all Java technology become closed source. Fortunately, common sense prevailed and was made the open version that I use daily as 90% of the developers. I think the best thing is not to generate constraints, but make our environment "flexible", ie that it can use either format, depending on the constraints present in a given country. I do not like to think that a user embargoed, such as an Iranian person, FGFS can not use because the DDS is part of a proprietary technology which can be banned for export.
In my work it is normal to try to make the system flexible. In this case I think it is better to write a few hundred lines of code in more and introduce a scheme like the one I included in the diagram:

DDS-PNG-JPG Automatic format convert

The idea is simple, you can use various formats, such as SVG which is also very useful for liveries, when the format is read a process transforms it into the format you want to use for your graphics card, such as the DDS , PNG or S3TC, the transformation could also use an appropriate pixel density as a function of monitor used! The everything is sent to a cache that can only be in RAM or, as I often do, mixed RAM-HD according to the type of use. This approach allows to minimize the load of conversion and at the same time allows to have the images already ready for use. Normally a pilot flies in areas fairly restricted and then the cache may already contain all what interests him limiting in this way the load of the CPU. An index, if the cache is mixed RAM-HD can handle various things, such as whether a file has been updated and then redoing the conversion, manage LIFO etc ... This seems like a solution is not too complicated, but extremely flexible, which separates the authors of the texture from the problems of format and its licenses. When the cache is stabilized, the CPU load is very small compared with its other activities. Fact that I have always occurred when I implemented these features in Java. --Abassign (talk) 09:11, 6 September 2014 (UTC)

about svg usefull for liveries: I'm not sure about. Of course, with canvas it would allow for new features (and I have many ideas, what can be done with... ;-)).

But svg-files aren't that cheap like a simple bitmap. I fear the more objects a svg-file has, the more computer ressource it will need. And a detailed livery will need a lot of objects. With canvas I already see a decrease of fps, and I'm not sure that only non-optimized nasal-scripts are the reason for.

But for .png and .jpg -files your approach sounds good, and I'm not sure if even X-Plane already uses this approach.

Cheers

--HHS (talk) 10:30, 6 September 2014 (UTC)

Gijs can probably help clarify things a little here, he mentioned several times on the forum that he's been using SVG for liveries IIRC. Obviously, a SVG file has "layers", that need to be rasterized each - with Canvas that means, that each layer becomes a "group". SVG parsing does happen in scripting space, but should normally only happen during intialization and not at run-time. Otherwise, all the groups/layers are pre-created and put into C++ data structures (CanvasElements) and directly handled by ShivaVG.

If you have doubts about the performance of Nasal/Canvas please post your code so that we can take a look - if you're right and your code is sufficiently optimized, then we can still use the Built-in Profiler to see why your code is triggering slow Nasal/Canvas code paths.

Also, it would help us if you could tell us exactly what aircraft/features you're referring to with regard to "I already see a decrease of fps" - there are varying examples of Nasal/Canvas code in FG, and generally only TheTom's code is sufficiently optimized - code like the NavDisplay is still relatively unoptimized, and even more so the Avidyne Entegra R9 code is not particularly optimized but doing a ton of stuff in Nasal that must be slow. Even if you're right, Canvas being "slow" now doesn't have to mean that Canvas will remain "too slow" in 2-3 release cycle.

Tom has already optimized several things, and I am sure we'll see more optimizations. But just making generic statements and stating assumptions obviously doesn't help improve the Canvas system. As could recently be seen on the devel list, we've had a fairly massive issues in the effects subsystem for years, tremendously slowing down effects (and FG as a whole) and even pointing at other subsystems - well informed bug reports are usually also dealt with, but just making generic claims isn't really helping anybody to check the merits of such statements.

Thus, it would make sense to use a heavily down-stripped test case, such as texturing the ufo this way to exclude other Nasal code from showing up in the profile. Let us know if you need help doing that. Thanks --Hooray (talk) 14:36, 6 September 2014 (UTC)

I use SVG as well for liveries or instruments, like gooneybird and many others. SVG is a scalable vector based graphics, so we can easily create liveries of different sizes without loss of quality, and compared to GIMP or Photoshop even with much finer quality. That's why I prefer paintkits with SVG over GIMP-paintkits.

But we still convert them to PNG for use in FGFS. SVG does not only have layers. Each line, cubes, circles etc. are objects aka elements. The more you have the more computer power it needs, at least when creating a SVG-file.

That's my concern about, that a complex SVG-file drawn in FlightGear needs a whole lot of computer ressource - completely independant of the underlying nasal-script or anything else.

But it is still an assumption and nothing I could proven yet.

--HHS (talk) 20:24, 6 September 2014 (UTC)

@HHS: Like I said previously: processing/parsing SVG files is indeed relatively slow because the parser is implemented in Nasal, but the only thing svg.nas is doing is looking for supported elements and converting them into so called CanvasElements, mainly osgText and shivaVG paths via CanvasText/CanvasPath.

In other words, all the "heavy" stuff is done directly when parsing the file - people using complex SVG files will see that the loading overhead does show up, not just because of the Nasal-based parser, but also due to the fact that possibly a ton of drawing primitives are converted into properties and put into the main property tree.

That's basically where things can be (and sometimes ARE) pretty slow. This part isn't currently optimized very much. However, all that being said, once the Canvas tree is set up and resides in the FlightGear property tree, rendering complexity is mainly a function of the number of modifications, and the result can be dealt with like a conventional raster image (e.g. PNG).

We're using the canvas:/// syntax to even reference complex SVGs as raster images and set up a simple NxN grid of symbols that is to be used as a cache. Thus, regarding your concern, the question is mainly if the SVG file is permanently animated or if it's rather static ?

The "static" use case should be relatively straightforward to optimize for - back in the early days of the Canvas, we did notice in various profiling runs that static canvas were permanently redrawn, Tom confirmed this and recognized it as lowing hanging fruit to optimize the system - shortly afterwards, we ended up with a new property to "cache" a prebuilt Canvas and not redraw individual elements each frame (IIRC).

Back then, this helped optimize the taxiway layer quite significantly. Also, Tom mentioned various times that he's interested in improving canvas performance, especially the upcoming GUI will benefit from such use-case specific optimizations, simply because a GUI is relatively static when compared to a MFD.

What the ND/Avidyne code is doing to work around the initialization overhead is to preprocess all imagery during startup. And a number of people have also requested some kind of built-in caching scheme using SQLite - which would basically make it possible to make a canvas persistent and serialize it to a blob. Tom mentioned that he was hoping to work towards this post 3.2

In the meantime, you can use one of the SVG tutorials to load a complex SVG file and measure the time it takes to render the whole thing via the systime()/benchmark() APIs. But in comparison to most other Canvas use cases, a livery should be relatively static still. So I wouldn't be too worried about it. --Hooray (talk) 00:43, 7 September 2014 (UTC)

Excuse me, I'd insert the SVG format only as a simple example to understand the flexibility of my proposal. Canvas However I find it fantastic and definitely want to be taken to implement some instruments in the G91 we are doing. The basic concept is to clarify whether this proposal makes sense, and it may be the key to solving the problems of compatibility between formats. My proposal is an attempt to have the maximum flexibility in size of all the graphic objects in FGFs and make compatible the simulation environment of FGFs in any context. At this point I ask the question to those who can develop in C ++ for FGFs, what I'm proposing is practicable? And What I am proposing can solve the problem of non-free formats? --Abassign (talk) 19:29, 6 September 2014 (UTC)

The idea sounds nice, but converting a PNG to DDS or any other format needs already several seconds here on my computer. Looking at the large amount of textures we have and we need of course, it doesn't sound all to practicable to me. But I'm not sure if I understand your proposal correctly. I heard from another developer off-line, that FGFS is already doing something like that. But I'm sure that I have doubts in using SVG for liveries without converting them to PNG before.

--HHS (talk) 20:24, 6 September 2014 (UTC)

The method I described can be very powerful for example, if the input file has the same file format that will be used directly by the graphics card. For example PNG or DDS. Only in the case that the input file is of a different format, such as JPG, BMP, SVG (with due caution;)), GIF etc will be necessary to undertake a transformation (format conversion) which will have a cost of process time , but when the file has been converted will be cached (LIFO), will can remain as long as its use will be sufficiently frequent. It is a technique similar to that which is adopted from hard drives that have a cache memory made ​​with two or three levels. Admit then that to convert from SVG to PNG / DDS files could be very interesting because the SVG format is very compact and allows to have, for example, liveries with different resolutions depending on the distance of the plane, in this case could be possible a marked improvement in display quality. For example it would be possible to have a livery high resolution when you look closely at the plane, or a livery with lower resolution when the plane is placed at a larger distance from the observation point.--Abassign (talk) 21:53, 6 September 2014 (UTC)

Thanks for the clarification, but we have this feature already. OSG is already creating such different resolutions, it is called mipmaps. Tim Moore and Mathias Fröhlich can describe this feature better and more detailed than I can do.

--HHS (talk) 10:49, 7 September 2014 (UTC)

SVGs may seem heavier than PNGs simply because SVG files will usually not just contain "data" but instructions for assembling an image from various primitives - those instructions need to be processed by "something" to get an image. Thus, a conventional raster image already contains the expected result/outcome - while a SVG image doesn't contain any image at all, but merely a bunch of rendering instructions in some kind of XML dialect that need to be processed by the SVG parser. In the case of Canvas, we only support a subset of SVG and map those to OpenVG/ShivaVG primitives typically. So it isn't exactly "fair" to compare performance here - SVG files are using a completely different approach: When you create a raster image containing a circle in Photoshop/GIMP, your image editor knows how to render a circle and how to turn this into a raster image - A SVG file, in constrast, will merely contain a circle instruction. This is where perceived slowness will typically come from, in combination with the aforementioned SVG/XML parsing and property tree overhead, and this is also why SVG files will usually be so small in comparison to a raster image, and why SVG can deal with arbitrary resolutions.--Hooray (talk) 15:40, 8 September 2014 (UTC)

I'm still against using SVG for liveries. It is heavier, and would needs too much resources for just showing a livery. For dirt the shaders can be used, the only thing is that we can't select the reflection map in runtime. But I'm sure there must be another way than using Canvas here, though I admit the idea was obvious.(Once tried to use the alpha layer, but it had a lot of disadvantages)

Though, I think there is a better place to use SVG: imagine you are approaching an airport in FlightGear. Just another boring approach. Suddenly: Bumm!! A big red stain just in front of you on the window! Cracks beside the stain! Birdstrike!

Or imagine, you are sitting in the aircraft, waiting for departure. It is rainy day, and you can see how the drops are running down in their characteristic way on the window. And you can still see the rain outside! When you finally accelerate your aircraft you can see how the drops changes their way and speed, and then begin to disappear.

X-Plane already show all this animation and I'm always amazed how realistic it look, and I'm sure that this would be more appropriate for Canvas and SVG!

--HHS (talk) 20:12, 15 September 2014 (UTC)

Have you meanwhile done any testing ? Frankly, this was just an idea because of a few postings on the forum. All testing that I've done on sufficiently modern hardware suggests that SVG/ShivaVG isn't currently a bottleneck. As you have probably seen, I've meanwhile experimented with this a bit and posted code to demonstrate how to do this kind of thing using Nasal/Canvas.

Personally, I don't have any preference at all - but I also don't create liveries usually. And while shaders/effects are an obvious, and fast, method - not everybody has access to hardware that can run shaders.

Typically, The people suggesting to use shaders/effects for such things are usually "power contributors" with a lot of GPU power. And as we can currently witness on the devel list, the effects/shader framework comes with its own issues and challenges, memory leaks and race conditions being pretty serious showstoppers obviously. Canvas has basically no barrier to entry and it allows people to do these kinds of things without having to learn effects/GLSL coding.

It certainly cannot be compared to effects/shaders performance-wise, but that was never the point. Also, I am far from being opposed to shaders/effects - as you may have seen, we already started exploring effects/shader support to Canvas and expose Canvas textures to the effects system, which would allow people to do all kinds of fancy things without having to know much about C++ programming.

Given that two contributors do have code doing this kind of thing, and given Zan's work, it is pretty likely that we'll eventually see this being merged. Whenever I've talked to aircraft developers about such features, they primarily want thing to work sufficiently fast, without any special-case treatments for things like Rembrandt - Canvas/SVG can provide just that.

Overall, people will continue to use what works best for them - especially aircraft developers without a coding background will surely appreciate the simplicity of Nasal/Canvas here - all the things that you mentioned could be easily implemented in ~50-60 lines of Nasal code. But I am certainly not saying that we should discard effects/shaders, it's more about keeping hardware requirements in mind - and even if shader support is available, our scenery folks surely know how to make use of all the horsepower there...

Also, there are various scenarios that could greatly benefit from run-time scenery modification in the sense that Canvas supports it - i.e. anything that is combat/damage related (think f14b or flug's bombable). Equally, the same method can be used for placing runway skid marks and so on.

Conceptually, a Canvas property tree actually isn't much different from a SVG DOM - the main difference being that it is hardware-accelerated and using the verbose PropertyList XML format, but any Canvas can be expressed as a SVG tree and vice versa - thus, this kind of development is pretty natural.--Hooray (talk) 20:26, 15 September 2014 (UTC)