Switching default texture format to DDS: Difference between revisions

Jump to navigation Jump to search

Switching default texture format to DDS (view source)

Revision as of 23:19, 4 September 2014

35 bytes removed ,  4 September 2014
Copyediting; first section; spelling; cat: Core development projects; etc
(Copyediting; first section; spelling; cat: Core development projects; etc)
Line 1: Line 1:
{{Note|
The FG development team is considering to '''switch the default format for terrain textures from png to DDS'''. This would offer a number of significant advantages:
 
* DDS is a compressed format, hence the download size of the FG base package may be decreased
* compressed dds can be directly used by many graphics cards, reducing also GPU memory consumption
* DDS stores all texture resolution levels, i.e. no lower resolution levels have to be generated when the texture is used, hence it loads much faster into memory
* the resolution levels ('mipmaps') can be customized, allowing for some interesting effects at no performance cost
 
Practically all commercial simulations use DDS for these reasons.
 
== Feedback needed - should FlightGear switch the defaults to DDS format for terrain texturing? ==
 
However, the DDS compression algorithm is patented, which means that it is not readily available for open source graphics drivers used by Linux distributions. Dependent on the specific hardware, this may or may not be a problem (modern graphics cards typically do not need the driver to process DDS, for older graphics cards there are non-patented workarounds available which decompress the DDS on the software level). The development team is concerned about making the FlightGear experience pleasant for all users, hence we would like to gather feedback how many users would be affected by a change in practice.
 
{{FGCquote
{{FGCquote
   |let's collect some some feedback [on DDS usage in FlightGear] until late November and restart this topic. We probably know by then what we do for the next release. "Somebody" needs to collect the feedback, however.
   |let's collect some some feedback [on DDS usage in FlightGear] until late November and restart this topic. We probably know by then what we do for the next release. "Somebody" needs to collect the feedback, however.
Line 8: Line 20:
   }}
   }}
}}
}}
}}
{{WIP}}


== Feedback needed - should Flightgear switch the defaults to dds format for terrain texturing? ==
If there are no problems reported, FG will change defaults to textures in DDS format with the 3.4 release, and then phase out the use of png textures.


=== What is this about? ===
=== What we need you to do? ===


The FG development team is considering to switch the format for terrain textures from png to dds. This would offer a number of significant advantages:
FlightGear already provides the simple option to test a DDS texture set. If you are running on Linux and use an open source graphics driver, please take 5 minutes to help during your next FG session:


* dds is a compressed format, hence the download size of the FG base package may be decreased
# Open the dialog under View -> Rendering
* compressed dds can be directly used by many graphics cards, reducing also GPU memory consumption
* dds stores all texture resolution levels, i.e. no lower resolution levels have to be generated when the texture is used, hence it loads much faster into memory
* the resolution levels ('mipmaps') can be customized, allowing for some interesting effects at no performance cost


Practically all commercial simulations use dds for these reasons.
# Under 'Terrain texture scheme', change the default 'Region-specific' to 'Global alternative (DDS format)'


However, the dds compression algorithm is patented, which means that it is not readily available for OpenSource graphics drivers used by Linux distributions. Dependent on the specific hardware, this may or may not be a problem (modern graphics cards typically do not need the driver to process dds, for older graphics cards there are non-patented workarounds available which decompress the dds on the software level). The development team is concerned about making the Flightgear experience pleasant for all users, hence we would like to gather feedback how many users would be affected by a change in practice.
# Press 'Okay' - FG will reload the terrain


If there are no problems reported, FG will change defaults to textures in dds format with the 3.4 release, and then phase out the use of png textures.
# Do you see proper textures on the terrain (they may look different and may also not fit the location perfectly)? If yes, you're fine. If you see monochromatic colors or other rendering artifacts, your system may have problems with DDS.


== What would we need? ==
# Change back to the texture scheme you like best


Flightgear already provides the simple option to test a dds texture set. If you are running on Linux and use an OpenSource graphics driver, please take 5 minutes to help during your next FG session:
# Enter your experiences in the list below
 
* Open the dialog under View -> Rendering
 
* Under 'Terrain texture scheme', change the default 'Region-specific' to 'Global alternative (DDS format)'
 
* Press 'Okay' - FG will reload the terrain
 
* Do you see proper textures on the terrain (they may look different and may also not fit the location perfectly)? If yes, you're fine. If you see monochromatic colors or other rendering artifacts, your system may have problems with dds.
 
* Change back to the texture scheme you like best
 
* Enter your experiences in the list below


Thanks for your help!
Thanks for your help!


== Tested hardware and graphis drivers ==
== Tested hardware and graphics drivers ==
{{Note|Please add your own hardware/results here !}}


=== NVIDIA proprietary driver ===
=== NVIDIA proprietary driver ===
Line 88: Line 81:
|}
|}


=== NVIDIA OpenSource driver ===
=== NVIDIA open source driver ===


=== Intel proprietary driver ===
=== Intel proprietary driver ===


=== Intel OpenSource driver ===
=== Intel open source driver ===
{| class="wikitable sortable"
{| class="wikitable sortable"
!Card
!Card
Line 123: Line 116:
|}
|}


=== ATI/AMD OpenSource driver ===
=== ATI/AMD open source driver ===
{| class="wikitable sortable"
{| class="wikitable sortable"
!Card
!Card
Line 177: Line 170:
is allowed to implement compression/decompression code for these image formats. Even if this is easy implementation wise.
is allowed to implement compression/decompression code for these image formats. Even if this is easy implementation wise.


I do not care for the on disk format of any image file we have. But the problem  is that some kind of precompression that can be stored in these dds files  
I do not care for the on disk format of any image file we have. But the problem  is that some kind of precompression that can be stored in these DDS files  
cannot be used with other drivers than the closed ati and nvidia ones. As long as these patented compression techiques are not used, every OpenGL  
cannot be used with other drivers than the closed ATI and NVIDIA ones. As long as these patented compression techniques are not used, every OpenGL  
driver can use this and displays this fine.
driver can use this and displays this fine.


Think: Intel has the hugest marketshare in graphics today. If I remember right, they sell more than ati and nvidia together (*). This kind of change in  
Think: Intel has the hugest marketshare in graphics today. If I remember right, they sell more than ATI and NVIDIA together (*). This kind of change in  
effect rules out the majority of users as intel cannot work with these files.
effect rules out the majority of users as Intel cannot work with these files.


(*) There are several statistics out there, this is the intel one that counts all sold computers. AMD does usually counts the revenue made with graphics  
(*) There are several statistics out there, this is the Intel one that counts all sold computers. AMD does usually counts the revenue made with graphics  
boards (where ati wins I believe) or nvidia that usually limit statistics to professional systems (where nvidia wins).
boards (where ATI wins I believe) or NVIDIA that usually limit statistics to professional systems (where NVIDIA wins).


I would like to have a flightgear that is by default just running on every average system. Having this run faster on a special configured system with some  
I would like to have a FlightGear that is by default just running on every average system. Having this run faster on a special configured system with some  
better configuration options and hand tuned hardware and drivers is very fine. But without tuning it must at least work in an acceptable way.
better configuration options and hand tuned hardware and drivers is very fine. But without tuning it must at least work in an acceptable way.


I have checked in a change to flightgear to make the use of the compressed internal formats a starttime configuration option.
I have checked in a change to FlightGear to make the use of the compressed internal formats a start time configuration option.
I am still interrested if we have that hangs also with texture compression disabled and without providing precompressed dds textures?
I am still interested if we have that hangs also with texture compression disabled and without providing precompressed DDS textures?


this is the reason for the (warning) message. If your machine would refuse to display this you, developing that, would probably just say 'nice try, but it does not work' before you check in something. In the case it displays fine, you probably say 'it works here, so I assume it works also for others, lets do'.
this is the reason for the (warning) message. If your machine would refuse to display this you, developing that, would probably just say 'nice try, but it does not work' before you check in something. In the case it displays fine, you probably say 'it works here, so I assume it works also for others, lets do'.
Line 202: Line 195:
not like this. You will just see this in the best case as the models look wrong, and in the worst case fgfs just crashes the driver if these textures  
not like this. You will just see this in the best case as the models look wrong, and in the worst case fgfs just crashes the driver if these textures  
are used. What I really care about is that these files are expected not to work on a huge  amount of graphics boards out there. The point is to tell people doing  
are used. What I really care about is that these files are expected not to work on a huge  amount of graphics boards out there. The point is to tell people doing  
textures that they should omit compression so that this message disapears.  
textures that they should omit compression so that this message disappears.  


The motivation behind this mentioned change is to sort out the best compromise so that the hangs hopefully disappear - which I hope to get with precomputed  
The motivation behind this mentioned change is to sort out the best compromise so that the hangs hopefully disappear - which I hope to get with precomputed  
mipmaps - and being able to display fine with any driver/gpu.
mipmaps - and being able to display fine with any driver/GPU.




Line 217: Line 210:
The previous strategy to have on disk an format that everybody can read and to make the driver compress them as needed/possible is better I think?
The previous strategy to have on disk an format that everybody can read and to make the driver compress them as needed/possible is better I think?


I hope that we can get rid of the compression. Can somebody with the apropriate tools convert the compressed textures to non  compressed ones? That could still be dds, but dds without these compressions that produce the warning. So no problem with cubemaps in dds as long as the  
I hope that we can get rid of the compression. Can somebody with the appropriate tools convert the compressed textures to non  compressed ones? That could still be DDS, but DDS without these compressions that produce the warning. So no problem with cubemaps in DDS as long as the  
compression is not there. Then *everybody* is again able to use this stuff.
compression is not there. Then *everybody* is again able to use this stuff.


* Just do not use the patented compression stuff. The precomputed mipmaps could probably do the job of avoiding the hangs (hopefully? to be checked?). May be we could lower disk space usage by providing a dds.gz or similar wrapper?
* Just do not use the patented compression stuff. The precomputed mipmaps could probably do the job of avoiding the hangs (hopefully? to be checked?). May be we could lower disk space usage by providing a dds.gz or similar wrapper?


* If it's just the mipmaps. May be we can precompute the mipmaps using the cpu in the database loader thread. This would help all textures not only the ones that could be converted. May be this is the most generic solution.
* If it's just the mipmaps. May be we can precompute the mipmaps using the CPU in the database loader thread. This would help all textures not only the ones that could be converted. May be this is the most generic solution.


* Implement some kind of image lookup order that knows if the compressed files could be handled or not. On loading an image in case of available compression  
* Implement some kind of image lookup order that knows if the compressed files could be handled or not. On loading an image in case of available compression  
first try to find a dds file with the same name of the original one. That involves some 'magic' which often leads to problems but that could at least work.
first try to find a DDS file with the same name of the original one. That involves some 'magic' which often leads to problems but that could at least work.


Other ideas? Also may be creative ones?
Other ideas? Also may be creative ones?
Line 234: Line 227:
points made by Mathias in 2012[https://www.mail-archive.com/flightgear-devel%40lists.sourceforge.net/msg37848.html]:
points made by Mathias in 2012[https://www.mail-archive.com/flightgear-devel%40lists.sourceforge.net/msg37848.html]:


Could we do dds files without compression but with precomputed mipmaps?
Could we do DDS files without compression but with precomputed mipmaps?


So at next, can you try out which combination of compression/provided mipmaps/forced simgear compression still work fine?
So at next, can you try out which combination of compression/provided mipmaps/forced simgear compression still work fine?


"precompute the mipmaps in the loader. IIRC tests with some of the guys suffering from this problem, providing premipmapped but uncompressed dds files had helped to get a fluent viewer. The argument against providing these dds files in general was that these files are really huge because of not including any compression and having all the  mipmaps. But that means we could at the point where the warning happens compute the  mipmap levels on the cpu in the loader thread. osg::gluScaleImage could be  
"precompute the mipmaps in the loader. IIRC tests with some of the guys suffering from this problem, providing premipmapped but uncompressed DDS files had helped to get a fluent viewer. The argument against providing these dds files in general was that these files are really huge because of not including any compression and having all the  mipmaps. But that means we could at the point where the warning happens compute the  mipmap levels on the CPU in the loader thread. osg::gluScaleImage could be  
used to do this I think (or something similar not requireing a context). This one is an imported version of the original glu function that is included in  
used to do this I think (or something similar not requiring a context). This one is an imported version of the original glu function that is included in  
osg for running on an EGL stack which no longer provides GLU. That is take the image scale this to the 1st mipmap, take the 1.st mipmap and scale this to the 2. mipmap and so forth.
OSG for running on an EGL stack which no longer provides GLU. That is take the image scale this to the 1st mipmap, take the 1.st mipmap and scale this to the 2. mipmap and so forth.
This would have the advantage that the png's do not deviate from the dds files over time."
This would have the advantage that the png's do not deviate from the DDS files over time."


computing mipmaps for any texture file on the CPU in the loader thread should globally improove the situation.
computing mipmaps for any texture file on the CPU in the loader thread should globally improve the situation.
Also avoiding the compression already in the files should help every use case. Except that the on disk memory consumption is higher. Well and except that the database loader has more work to do on the CPU.
Also avoiding the compression already in the files should help every use case. Except that the on disk memory consumption is higher. Well and except that the database loader has more work to do on the CPU.


Line 249: Line 242:
For the disk usage, I think gzip compressing these will work sufficiently fine. Ram usage of the images should not hurt too much. Sure the images are bigger  
For the disk usage, I think gzip compressing these will work sufficiently fine. Ram usage of the images should not hurt too much. Sure the images are bigger  
in memory. But fgfs is not just about images - far from that. On the GPU, you can still use compression for the textures as the internal  
in memory. But fgfs is not just about images - far from that. On the GPU, you can still use compression for the textures as the internal  
format. This is what flightgear tries to do if the extension is supported (checked by osg).
format. This is what FlightGear tries to do if the extension is supported (checked by OSG).


The major point is that there are several ways that use slightly more resources to get around this problem.
The major point is that there are several ways that use slightly more resources to get around this problem.
But once the patented compression is on disk, there is *no* way back for people not having this feature.
But once the patented compression is on disk, there is *no* way back for people not having this feature.


If you have better ideas that do not rule out intel and the oss drivers, you are welcome!
If you have better ideas that do not rule out Intel and the OSS drivers, you are welcome!


On linux this should work as long as you have zlib installed which could be regarded as a system library there. Can we rely on zlib being compield into our osg distributions on win32? We do  need zlib in any case, it's mostly about teaching osg that it finds our zlib on configure and build the gz plugin.
On linux this should work as long as you have zlib installed which could be regarded as a system library there. Can we rely on zlib being compiled into our OSG distributions on win32? We do  need zlib in any case, it's mostly about teaching OSG that it finds our zlib on configure and build the gz plugin.


FredB implemented a mipmap control and generation tool in effects when he last updated the urban shader. For the moment, it relies on hardware when the average  operator  is used for all texture channels but it could easily be modified to compute all mipmap on the CPU. look the <mipmap-control> effect option and  
FredB implemented a mipmap control and generation tool in effects when he last updated the urban shader. For the moment, it relies on hardware when the average  operator  is used for all texture channels but it could easily be modified to compute all mipmap on the CPU. look the <mipmap-control> effect option and  
Line 447: Line 440:
   }}
   }}
}}
}}
[[Category:Core development projects]]
Retrieved from "https://wiki.flightgear.org/Special:MobileDiff/75903"

Navigation menu