Talk:Building FlightGear - Cross Compiling

Build Errors

is it a good idea to update the page with the build errors encountered and the possible solutions(like for example with OSG) so as to make the status more informative? -- Hamzaalloush (talk) 18:22, 18 May 2015 (EDT)

sure, why not - we should add anything here that may help others at some point, even if it's just for historic reasons, i.e. the project having a history before it gets a visible commit log. Besides openscenegraph.mk still seems to contain a bunch of static-build related build flags as far as I can tell, are you sure that dynamic/shared builds are intended to be supported ?--Hooray (talk) 18:38, 18 May 2015 (EDT)

yes, you have to change the flags to build dynamically, and this is how you finally get shared libraries into /bin, else they get built into /lib with the *.a prefix as static. actually some shared libraries actually do get installed while some require linking with the plugins before encountering error , this is what i intend to work on -- Hamzaalloush (talk) 18:52, 18 May 2015 (EDT)

It is usually a good idea to document problems and how they got solved. If not for else then for to not do them again.

It could also potentially help similar endeavors later on.

—Johan G (Talk | contribs) 06:38, 19 May 2015 (EDT)

Required OSG Plugins

can we get a subset of required plugin needed by flightgear? we can simply disable the faulty ones of the time bieng if they are not needed? mxe allows you to apply patches to source from the *.mk file as well for this -- Hamzaalloush (talk) 18:58, 18 May 2015 (EDT)

sounds like a plan, I'd use grep -R and the OSG_USEPLUGIN macro in $SG_SRC and $FG_SRC to see what's important [1]. --Hooray (talk) 19:06, 18 May 2015 (EDT)

Minimizing the needed number of MXE packages

the default 'all-filtered' make target, builds 367 packages in a clean environment, only 107 out of 367 are built using MXE's own dependency walker in the Makefile if 'openscenegraph' is passed as a parameter to 'make'. few packages can be excluded by disabling uneeded OSG plugins, and dependencies of further packages. -- Hamzaalloush (talk) 21:16, 22 May 2015 (EDT)

mxe specific tool chain

as far as I know, mxe should be quite able to only build those dependencies that are needed for FG - i.e. plib/openal/simgear/osg which should in turn pick up their own dependencies automatically. So it's mainly a matter of making sure that dependency resolution is working correctly. If you are really referring to the underlying tool chain and not just dependencies, I am not sure what might be optional ? But I guess it would make sense to come up with 2-3 *.mk files for the main FG dependencies, so that the a corresponding build environment can be automatically created by the mxe installer, and which can be used by the Superbuild afterwards. --Hooray (talk) 13:55, 23 May 2015 (EDT)

as i experienced this works very well(95% of the time, MXE pulls only required packages), but due to some packages that introduce dependency bloat, such as for instance OSG compiles with ffmpeg(we don't need this), so this nice package pulls all the audio tools such as x264, flac encoders etc. we can minimize this in our fork, such that ffmpeg is disabled completley, further elimination of a dozen packages consisting of only encoders built for ffmpeg, i'm sure i can find other examples like this as well. FG main dependancies such as simgear will be included in the 'requirements' for fgfs.mk, so whenever you call 'make fgfs' it builds it. for the Superbuild, i'm thinking an extra automation step is required preferably by a script in $(MXE_ROOT)/tools that copies all the dll's and binaries to the expected Building_using_CMake_-_Windows#Directory_tree. -- Hamzaalloush (talk) 16:40, 23 May 2015 (EDT)

I do agree, I have also seen optional dependencies being "defaulted" to "yes" for the sake of simplicity - while ffmpeg isn't currently needed by SG/FG, a few FG devs (especially those doing UAV stuff) have been contemplating to support ffmpeg-based video streaming in FG at some point (see supporting streaming in FG for details/references), so I guess it would be better to identify such osg/mxe issues (I think some osg examples require ffmpeg), and make the corresponding dependency optional by adding proper dependency resolution, e.g. by using the corresponding OSG magic to detect such dependencies (osg-config, or even just cmake configuration flags). I do however agree that it makes sense to first get a basic build working, and only look at all the other optional stuff later on - we will need some kind of solution to deal with those anyway (think fgcom, profiler, osgEarth, HLA/RTI etc).--Hooray (talk) 13:49, 24 May 2015 (EDT)

if that's the case, i might ignore this for now and keep ffmpeg, unless it gives error and then i'll proceed with a basic build. in static-mingw, until now only one OSG plugin has been making problems which was when building 'gdal', which i assume necessary for osg-earth, I’ll write the specifics in the Issues/OSG topic when i'm done so we can resolve it later. -- Hamzaalloush (talk) 19:35, 24 May 2015 (EDT)

true, gdal is needed for osgEarth, but would also be useful for supporting mxe-based building of TerraGear --Hooray (talk) 06:55, 28 May 2015 (EDT)

3rd party deps/directory structure

I still need to check the Superbuild/CMakeLists.txt files, but I assume that those assumptions are not Windows specific, but primarily MSVC specific - so may not even be applicable to a gcc-based cross-compiler ? --Hooray (talk) 13:54, 24 May 2015 (EDT)

yes your right! :-) after i checked, this was to resolve MSVC linking to 3rd parties. that dir structure might not even be needed, still, i think it is a good idea to at least compile the respective versions of those 3rd party dependencies as provided in the fgmeta/3rdparty branch. -- Hamzaalloush (talk) 19:30, 24 May 2015 (EDT)

it definitely is a good idea, but looking at the two CMakeLists.txt in SG/FG, those 3rd party deps are there mainly for those systems not providing "system-wide" versions of those - so we could just as well review existing mxe *.mk files and directly add support for the equivalent libs to mxe, to ensure that there both options are provided. As far as I understand, system-libs would be chosen by default on *nix platforms anyway, so we could just as well ignore 3rd party deps when doing cross-compilation, by simply using the corresponding mxe equivalents for those. Overall, I think that "3rdparty.zip" is a workaround addressing shortcomings in the Windows build process - so if it is easier to directly build the proper packages as part of mxe, I would consider doing that instead, as it feels more "natural" to me. --Hooray (talk) 15:33, 28 May 2015 (EDT)

certainly this was my thought process recently too. as FG is known to compile against development version of those. so i will leave them as is, and probably even remove that objective from the roadmap. it makes sense to continue unless we have problems. -- Hamzaalloush (talk) 16:28, 28 May 2015 (EDT)

Testing & Development

it would be helpful if there was automation step involved, to routinely cross-compile crucial MXE element for current status, as for instance when certain MXE package "$(PKG)_URL" elements becomes unavailable upstream. sort of like a Jenkins implementation. -- Hamzaalloush (talk) 05:29, 27 May 2015 (EDT)

I've been in touch with a few contributors - unless we can get the FlightGear Build Server maintainers involved, we do have at least one volunteer to provide hosting on a VPS, where mxe and jenkins could be set up, with full root access provided as well. Alternatively, the OpenSuse Build Service (OBS) does provide Jenkins integration, too: https://en.opensuse.org/openSUSE:OpenStack_and_Crowbar_development_process

Given all the recent progress, and the increasing number of contributors wanting to help with different aspects of this effort, I do think that it would make sense to either register an OBS account, and/or, sign up for gcc compile farm access: https://gcc.gnu.org/wiki/CompileFarm - any preferences ? --Hooray (talk) 15:30, 28 May 2015 (EDT)

no i do not. which infrastructure is more easier to use? i think i can leave this decision in the hands of someone with more experience than me, so please feel free to suggest any ideas or someone to consult/contribute? -- Hamzaalloush (talk) 16:27, 28 May 2015 (EDT)

The "gcc compile farm" is just a bunch of dedicated servers reachable via SSH where you can download/build and run stuff, so fairly easy to use for people familiar with Linux - the OBS is a different beast, and pretty sophisticiated, too - i.e. it can even be used to automatically package up software for different distros automatically (think deb, ppa, rpm etc), and it comes with tons of support and documentation, too - while also being possible to integrate with Jenkins based CI servers and a few IDEs. --Hooray (talk) 16:50, 28 May 2015 (EDT)

OBS can be used for building/creating packages for pretty much all distros/package managers, and it will also handle hosting (which might be useful given that a binary mxe package may easily be ~3-4 gb in size). --Hooray (talk) 09:11, 30 May 2015 (EDT)

simgear.mk/fgfs.mk

i think i can prototype something right now for this, should i parallel with (FlyHigh -saiarcot895? what is his status right now? -- Hamzaalloush (talk) 16:35, 28 May 2015 (EDT)

having a prototype (for SimGear headless) would make sense in my opinion, and it coud be used as a template for other (non-headless) stuff, including both, SG & FG as a whole, but also other dependencies (think openrti). So given that SimGear is the only common dependency here, I would suggest to prototype a well-commented example and get this committed, so that others can take a look and learn from it, e.g. to help refine SG/FG support and come up with modules for optional dependencies. The Superbuild would be a different beast anyway, because it is trying to do everything (including the source check outs), so that would require some thinking - and having experience with dedicated simgear.mk and flightgear.mk files should come in very handy for that.--Hooray (talk) 16:47, 28 May 2015 (EDT)

I have a preliminary simgear.mk that has most of the dependencies and the building, but haven't reached testing stage yet because I'm having trouble compiling netcdf. -- Flyhigh (talk) 21:47, 28 May 2015 (EDT)

Well, I found at least half of the problem: portablexdr has an empty build command for x86_64-w64-mingw32, which I was building in. Also, I've uploaded the simgear.mk to here -- Flyhigh (talk) 22:21, 28 May 2015 (EDT)

yeah, some clean-up required for these x86_64-w64-mingw32 entries there. i suggest removing them as i did removing $(PKG)_BUILD_SHARED complety whenever i get a [no-build] result for shared package. nice work on getting the preliminary simgear.mk up, i'll test with it, but let me know when you are finished working that x86_64-w64-mingw32 toolchain to produce openscenegraph, please commit your progress for each package fixed. also, if you want i can merge that simgear.mk commit into my fork, or you do a pull request? whichever you prefer. i'll be following your repo as a remote. -- Hamzaalloush (talk) 04:22, 29 May 2015 (EDT)

I've switched to i686-w64-mingw32.shared for now, only because everything else is set up there. I'll test x86_64 once I have Simgear up and running. Simgear currently fails to get past CMake, so I'll be working on that today. -- Flyhigh (talk) 11:04, 29 May 2015 (EDT)

Building full (non-headless) Simgear now works. I've bumped up the progress to 50%, and once Flightgear is built, I'll set it to 100%. -- Flyhigh (talk) 14:16, 29 May 2015 (EDT)

great job, also set the 3rd party dependancies to 100% when your done with FG build. -- Hamzaalloush (talk) 15:16, 29 May 2015 (EDT)

Flightgear now builds and starts up (after a few changes to the Simgear build script), but crashes sometime around initializing scenery. Also, FgCom is disabled currently due to problems with libiax. Also, Simgear is built in C++11 mode, while Flightgear is built in the default (C++89 or C++99, I think) mode; this could be one of the reasons why Flightgear is crashing. -- Flyhigh (talk) 11:31, 30 May 2015 (EDT)

Superbuild

This isn't relevant at the moment, but referring to the Superbuild related comments above: it seems we could end up with a viable workaround by figuring out if/how and where those *.mk files are getting the value for -DCMAKE_TOOLCHAIN_FILE='$(CMAKE_TOOLCHAIN_FILE)' from - i.e. CMAKE_TOOLCHAIN_FILE specifically, and directly reuse that in the Superbuild (need to check if that'd work). But if it's too much of a hassle, let's just focus on *.mk files for starters. --Hooray (talk) 17:06, 28 May 2015 (EDT)

sure, it's stored in mxe/usr/${triplet-name}/share/cmake/mxe-conf.cmake, i'm sorry if this is not the place to include here, but here is it's contents anyway, autotools has similar implementation with $(MXE_CONFIGURE_OPTS), for autotools that is contained in the Makefile IIRC. mxe-conf.cmake: -- Hamzaalloush (talk) 17:14, 28 May 2015 (EDT)

set(CMAKE_SYSTEM_NAME Windows)
set(MSYS 1)
set(BUILD_SHARED_LIBS ON)
set(LIBTYPE SHARED)
set(CMAKE_BUILD_TYPE Release)
set(CMAKE_FIND_ROOT_PATH /home/hamza/build/hamza-mxe/usr/i686-w64-mingw32.shared)
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_C_COMPILER /home/hamza/build/hamza-mxe/usr/bin/i686-w64-mingw32.shared-gcc)
set(CMAKE_CXX_COMPILER /home/hamza/build/hamza-mxe/usr/bin/i686-w64-mingw32.shared-g++)
set(CMAKE_Fortran_COMPILER /home/hamza/build/hamza-mxe/usr/bin/i686-w64-mingw32.shared-gfortran)
set(CMAKE_RC_COMPILER /home/hamza/build/hamza-mxe/usr/bin/i686-w64-mingw32.shared-windres)
set(CMAKE_MODULE_PATH "/home/hamza/build/hamza-mxe/src/cmake" ${CMAKE_MODULE_PATH}) # For mxe FindPackage scripts
set(CMAKE_INSTALL_PREFIX /home/hamza/build/hamza-mxe/usr/i686-w64-mingw32.shared CACHE PATH "Installation Prefix")
set(CMAKE_BUILD_TYPE Release CACHE STRING "Debug|Release|RelWithDebInfo|MinSizeRel")
set(CMAKE_CROSS_COMPILING ON) # Workaround for http://www.cmake.org/Bug/view.php?id=14075
set(CMAKE_RC_COMPILE_OBJECT "<CMAKE_RC_COMPILER> -O coff <FLAGS> <DEFINES> -o <OBJECT> <SOURCE>") # Workaround for buggy windres rules
set(PKG_CONFIG_EXECUTABLE /home/hamza/build/hamza-mxe/usr/bin/i686-w64-mingw32.shared-pkg-config)
set(HDF5_C_COMPILER_EXECUTABLE /home/hamza/build/hamza-mxe/usr/bin/i686-w64-mingw32.shared-h5cc)
set(HDF5_CXX_COMPILER_EXECUTABLE /home/hamza/build/hamza-mxe/usr/bin/i686-w64-mingw32.shared-h5c++)
set(Boost_THREADAPI "win32")
set(QT_QMAKE_EXECUTABLE /home/hamza/build/hamza-mxe/usr/i686-w64-mingw32.shared/qt/bin/qmake)

thanks, you spared me quite a bit of searching here - I think this should already be a good way to play with getting the Superbuild to work at some point, without it necessarily having to invoke/involve mxe *.mk files. Basically, we would need the equivalent of some kind of mxe-config script that provides the corresponding path for such keys (e.g. the CMAKE_TOOLCHAIN_FILE) so that this can be reused elsewhere, without having to hard-code any paths.--Hooray (talk) 17:19, 28 May 2015 (EDT)

hi, so all you need is an provoking script that initialize that file/set var as a standalone? And then it should be straightforward to adopt for the supper build? Then what happens to this project, is this just an exercise for cross-compiling FG? Or merely an extension to the Super build like d&c currently -- Hamzaalloush (talk) 11:39, 29 May 2015 (EDT)

even "just" *.mk based cross compilation would be useful already - however, it would obviously be platform specific, while the Superbuild is pretty much guaranteed to stay around, and to be maintained in the future - i.e. due to it not being specific to any single platform/os or compiler, and due to it being used by the Jenkins build server. All the dependencies would still need to build/work correctly - so I guess it would be more difficult to use/adapt the Superbuild directly. Once mxe is able to build FG and all deps using *.mk files, we should strive to review all necessary additions/patches and try to get those merged upstream (mxe, sg, fg etc) - the Superbuild would be nice to support because that would mean that mxe-based cross compilation would be handled by the same front-end. But even just having a handful of *.mk files would be very useful, i.e. to provide FG/osgEarth binaries to Windows-based end users.--Hooray (talk) 12:35, 29 May 2015 (EDT)

ok Hooray, i'll probably do a miniture adoption of building a small package with mingw64 using the requirements learned from $(MXE_CONFIGURE_OPTS) in the superbuild, do you have any progress? i'm looking for something to do meanwhile saiarcot895 are building for Flightgear static. -- Hamzaalloush (talk) 20:44, 3 June 2015 (EDT)

Project Focus

which is exactly why we started this whole initiative, to provide osgEarth binaries first and foremost. now, on the subject of getting these package changes reviewed and merged into upstream/master... i don't think that's realistic, given that i did my own thing to make it work *on my distribution*, so as it stands, for the sake of progress, we should focus on our branch, it's very clear to see how each plugin/dependancy was built from .mk files to check for regressions, my fork is so off the upstream it's not even funny, converting some packages to CMake and reverting to stable GCC among others. -- Hamzaalloush (talk) 20:16, 29 May 2015 (EDT)

ok, I see - I was mainly referring to required OSG/SG and FG level patches to ensure that those are merged (think CMakeLists.txt stuff). Regarding mxe, I would suggest to gather all related patches/branches and establish this as a "fork" intended to support compilation of OSG-based applications like FlightGear, which would mean that other OSG projects/developers may sooner or later be interested in using such an "mxe.osg" fork. Equally, I would hope that using the OBS would help us do some more testing on non Ubuntu distros eventually (I am also on Ubuntu 14.xx) to ensure that "mxe.osg" works across a handful of main distros. For now, the focus should probably be coming up with an osgearth.mk module to see if the OSG integration is able to link other OSG apps, i.e. SG/FG afterwards. Once that is the case, we can review poweroftwo's batch file and create separate simgear-osgearth.mk and flightgear-osgearth.mk files for FG/osgEarth. The Superbuild shouldn't be a priority right now, but I will make sure to check what is required to get that working. Thanks so far everybody, great job !--Hooray (talk) 09:00, 30 May 2015 (EDT)

Based on what I've seen, it probably will be possible to merge it into upstream (the commits might need some cleaning up), but the main blocker here is that the bug in GCC 5.1 that's blocking the compilation of OSG (or was it another library?) needs to be fixed. -- Flyhigh (talk) 11:27, 30 May 2015 (EDT)

it was for BOOST not able to link with stdlibc++ in GCC 5.1 when trying to define the std::codecvt_byname facet. see, [2] [3] -- Hamzaalloush (talk) 11:47, 30 May 2015 (EDT)

Well, I do agree that it will be easier to continue "as is" and just maintain a fork of mxe with a focus on building OSG-based applications like FlightGear, and maybe we can revisit getting mxe related changes merged upstream once we have accomplished our main objectives/priorities (i.e. being able to cross-compile FG/osgEarth on Linux for Windows using a set of downloaded deb/ppa packages) ? Once that is the case, most of the difficult work should be done, and we can look at which changes/patches have a chance of getting merged upstream. I also guess that our priorities may differ a bit afterwards - personally, I would be interested in 1) seeing the Superbuild support mxe, 2) creating a virtual appliance with "mxe.osg" pre-installed, and 3) adding better diagnostics to help Windows-users provide better bug reports (even though the latter may become more important for development/testing purposes). At that point, it should also be possible to either get in touch with the Jenkins maintainers to set up mxe.osg there, or set up our own Jenkins instance for CI purposes - e.g. using OBS or the gcc compile farm. So I think this is more about sharing our goals and identifying overlapping stuff to see how to proceed once SG/FG build/link and work correctly - for instance, some Windows folks would undoubtedly be able to use a binary fgfs.exe file "as is", while others may need an installer/patch set (binary overlay) - equally, for FSWeekend/LinuxTag-like events it would be awesome to have a self-contained statically linked fgfs.exe file that can be put on a USB drive/CD image. So there are many areas that may sooner or later benefit from this. Personally, I think that better diagnostics integrated within the binary would tremendously help developers, especially given the large number of Windows users, and those wanting to use FG/osgEarth. --Hooray (talk) 11:50, 30 May 2015 (EDT)

Debugging

i'm not very familiar with how do a backtrace on a Windows binary, other than using exception traps with the Windows SDK for driver errors, or using process monitor to check the stack and system calls. so i'm unsure how to provide this improvement in debugging facilities. i also have no experiance with CMake superbuild's, so not sure how to adopt this as a developer for the effort. in time, there will be more collaboration involved to get this done i'm sure, but right now it's good that you let me know about your overall goals. and it's good that you share them with me and what you want me to do, to my ability(which in the mean-time is only involving *.mk files), there is no conflicting goals, just different ambitions/priorities. -- Hamzaalloush (talk) 12:10, 30 May 2015 (EDT)

MXE can build GDB for Windows, which will help a little (at least for users familiar with debugging in Linux/Unix) in debugging. It works fairly well; the only issue I've encountered so far is that you can't interrupt a running program using Ctrl-C. That being said, though, these builds don't have any debugging symbols in them, so if someone wants to use GDB, they'll need to modify the .mk files. -- Flyhigh (talk) 12:23, 30 May 2015 (EDT)

I don't think there's any conflicting priorities so far - the backtrace can be provided by gdb (assuming the binaries/libs have been built with debugging symbols included). As far as I remember, WINE and gdb can even be coupled to do this automatically to some degree. But overall this is touching on my recent additions to the "Goals" section in the main article, i.e. the whole "BreakPad" idea we've been tossing around for a while. I still need to see how difficult it would be to add a breakpad.mk file and get the demo/examples working. Once that works, I can modify bootstrap.cxx and main.cxx in $FG_SRC/Main to ensure that backtraces are provided automatically during segfaults. The CMake superbuild shouldn't be a priority at the moment, I even think that adding better diagnostics to the Windows binary is more important - the Superbuild would be nice to support at some point, because it will be maintained in fgmeta, i.e. by the main project. I have been in touch with two more Linux/Windows-based contributors who are interested in helping with testing/developing this further. But for that, it would make sense to combine patches from both branches/forks and agree to use only a single repository, and provide commit access to all interested parties. If your main interest is getting osgEarth & sg/fg working, I suggest to focus on the corresponding *.mk files, and maybe take a look at poweroftwo's batch file (as per the thread on the forum). My suggestion would be to use a separate set of flightgear-osgearth.mk and simgear-osgearth.mk files, based on the main simgear/flightgear.mk files, once those are working. Regarding debugging symbols, I mentioned this elsewhere, I would suggest to focus writing *.mk files with -DCMAKE_BUILD_TYPE=Debug or at the very least RelWithDbg, instead of "Release" - mainly for testing/development purposes. By the way, please do feel free to add your own priorities/objectives to the main article, too - we can also add a corresponding table to provide a more detailed overview. --Hooray (talk) 12:31, 30 May 2015 (EDT)

Roadmap & Priorities

I've added a "People" column to the corresponding roadmap tables so that people interested in anything particular can easily add new items and associate roadmap items with the corresponding contributors, which should also help others to get in touch with the right people if necessary. --Hooray (talk) 12:52, 30 May 2015 (EDT)

thank you for all that you'v done Hooray, we appreciate it!! let's make this an example of complete wiki collaboration :) btw your welcome to add yourself to the developers!!

.deb packaging

what should the .deb package consist of? a freeze of a successful build with all it's packages? it's totaling 3 GB's right now on my system. -- Hamzaalloush (talk) 16:35, 28 May 2015 (EDT)

right, I guess we do have different target audiences: those just wanting to cross-compile FG (without all the setup overhead) would definitely benefit most from everything pre-compiled included, so I would suggest to keep things simple for now and just turn the whole shebang into a deb/ppa package to ensure that FG (including all deps) can be built. I can also help and turn this into a virtual appliance for VirtualBox (possibly including a working fg source tree) But I think we only need to look into this once the other items can be scratched off from the roadmap - i.e. once SG/FG actually build & link correctly. In the meantime, it should probably be safe to assume that all parties interested in this, are able to get/set up and use mxe from your github branch ? --Hooray (talk) 16:44, 28 May 2015 (EDT)

yup, you can clone this from my branch and get it going with only 'git clone -b master-oldgcc https://github.com/hamzaalloush/mxe-clone.git' && 'make openscenegraph', but i do not guarantee all packages will be working on non-Ubuntu system. i have a back-up rig that has Debian Wheezy in it, and was unable to compile with it. -- Hamzaalloush (talk) 16:50, 28 May 2015 (EDT)

this can be put on hold for now - it seems mxe is being cleaned up to provide native support for building binary packages.--Hooray (talk) 07:40, 2 November 2015 (EST)

Reported Segfaults

Can you provide a backtrace ? what about the other programs/unit tests (e.g. in SG: via make tests) - do they work correctly, or not ? (also, was this on an actual Windows installation or just using WINE/qemu?) For testing, you can also use various binaries in /utils (e.g. fgviewer)--Hooray (talk) 11:28, 30 May 2015 (EDT)

If we reach the stage where we succeed building/linking everything, but keep getting segfaults, we should definitely investigate prioritizing debugging support (e.g. using breakpad or some other mechanism that works sufficiently well (winedbg?)). --Hooray (talk) 09:25, 11 June 2015 (EDT)

MXE central repository

now as it stands, we have two seperate repo's, one for Flyhigh and the other for myself, so to not make any conflicting merges, do we create a new one? or do we adopt Flyhigh's? as it seems to be more complete than mine. -- Hamzaalloush (talk) 14:30, 30 May 2015 (EDT)

right, for now, I guess it makes sense to agree on a single repo, and add volunteers/contributors as committers - while asking people to help maintain the wiki article, so that others can more easily follow everything. I don't have any preference regarding which repo to use, so I trust your judgement. Besides, how long does it take for you to build everything ? Should we already look around for compile farm/OBS access (those systems typically having at least ~12 cores and ~32gb of RAM) ? --Hooray (talk) 14:35, 30 May 2015 (EDT)

i'm ready to remove my repository and base my work on Flyhigh's if he's ok, we are already done with FG, the mk files are there. even if not (seg-fault free). Flyhigh gonna have to report if it was by C++11 compilation(else might be core issue) [...] -- Hamzaalloush (talk) 14:51, 30 May 2015 (EDT)

ok, sounds like a plan --Hooray (talk) 15:06, 30 May 2015 (EDT)

ccache & distcc

as for the other question, it takes about 2 hours on an i5. we can look to the compile farm when doing testing other distribution i guess. -- Hamzaalloush (talk) 14:51, 30 May 2015 (EDT)

btw, on a single computer you can use ccache to cache previously compiled object files (which will take up several gb of disk space), if you have access to other computers on the LAN, you can also use distcc (possibly in conjunction with ccache). For me, using just ccache helps speed up OSG/SG/FG compilation significantly, because most files are unlikely to have been modified at all - so can usually be reused "as is". --Hooray (talk) 15:03, 30 May 2015 (EDT)

OSG & osgEarth

Given that OSG now builds correctly, I would suggest to come up with a dedicated osgearth.mk file for testing purposes, which we can also use as a template for SG/FG flightgear-osgearth.mk afterwards, while also preparing support for the osgearth branch poweroftwo has been maintaining. In addition, osgearth comes with a bunch of osgearth applications that should be useful for testing purposes (OSG vs. osgearth support, both being required for linking) --Hooray (talk) 00:32, 29 May 2015 (EDT)

good post-coffee (+3 GMT) afternoon :) sure, i'll look into that since Flyhigh is unto the sf/fg mk files for now. -- Hamzaalloush (talk) 04:25, 29 May 2015 (EDT)

it remains for me to make the osg-earth.mk(have some undefined references right now), than to get FG/SG with osgearth-patch. see [4], that'll be a matter of download and compile only as poweroftwo has rebased his work on release/3.4, and that's already working for us now. -- Hamzaalloush (talk) 14:51, 30 May 2015 (EDT)

dcmtk Static Build

I was trying to do a static build of Flightgear, which, along the way, needs dcmtk. However, it's searching for Wine and the C++ runtime, which is failing for me. How did you get past that? -- Flyhigh (talk) 21:39, 1 June 2015 (EDT)

did you try to revert to autotools version[5]? it works for i686-w64-mingw32.static target(just tested now). for i686-w64-mingw32.shared support, i did CMake conversion [6], and apply patch for finding 3rd party deps since dcmtk wasn't using FIND_PACKAGE in CMake/3rdparty.cmake, because it assumes we are on Windows [7]. my suggestion is to revert to autotools -- Hamzaalloush (talk) 09:26, 2 June 2015 (EDT)

Ok, so I did revert to the autotools version, but was wondering if the CMake version worked for you for a static build. I'm now getting an unusual linker error in building netcdf. It can't find a reference to, for example, _imp__SDfileinfo, but _SDfileinfo is present in libmfhdf.a. Any clue what might be going on here? I'm suspecting it has something to do with the symbol visibility. - Flyhigh (talk) 11:21, 2 June 2015 (EDT)

references to _imp suggests that it's looking for a 'shared' import library symbol. example ./lib/libmfhdf.dll.a, which does not exist in your tree. so find out why netcdf is linking with shared references? see: http://pastebin.com/FkF2nFRS -- Hamzaalloush (talk) 12:41, 2 June 2015 (EDT)

porting to GCC 5.1

well, there has been some progress to the missing exports for libstdc++, one of GCC's main developers has started arranging a patch after few other reporting with successful attemps freezing of the missing functions to an older version. a few other issues have arrisen after more investigation for this bug, so should we port back to GCC 5.1 when the patch is complete? is it worth it? https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66030 -- Hamzaalloush (talk) 20:21, 3 June 2015 (EDT)

MXE has committed a patch to fix building with GCC 5. I haven't tested it yet, though. Currently, I'm working on upstreaming the changes here (slowly), so that we can then work on the main upstream branch (with GCC 5 compatibility). -- Flyhigh (talk) 22:35, 4 June 2015 (EDT)

I should have noticed the following beforehand: 534, 611, 612 -- Flyhigh (talk) 23:27, 4 June 2015 (EDT)

alright, new update, we managed to get GCC 5.1 fixes in order to compile Boost without reverting. thanks. https://github.com/mxe/mxe/pull/750 http://lists.nongnu.org/archive/html/mingw-cross-env-list/2015-07/msg00001.html -- Hamzaalloush (talk) 20:23, 5 July 2015 (EDT)

Sorry, I somehow must have missed this update - I am just catching up with things... I am about to review the diffs and pull the latest changes. Thanks again for all your work (both of you!). I guess, that still leaves us with those segfaults, right?--Hooray (talk) 14:01, 8 July 2015 (EDT)

latest GCC changes haven't been merged into Saiarcoat branch for SG/FG i think, because his are rebased on my old fork with GCC 4.9. i haven't investigated with FG yet, but i'm making progress to compile Terragear every now and then. i think the current plan is to submit fixes for dependencies on upstream slowly. then resume to work on SG/FG seg-faults. -- Hamzaalloush (talk) 21:11, 8 July 2015 (EDT)

Status 06/2015 ?

Hi guys, what's the latest news/status, and where do we find your changes/commits ? And what is currently missing/broken, i.e. needs investigating ? --Hooray (talk) 11:34, 6 June 2015 (EDT)

Haven't gotten anywhere on the crash with i686 shared, so I'm working on the static side of things and also slowly upstreaming our changes here. Also, 64-bit is starting to be painful (mainly because of portablexdr). I'm now building i686 static and hoping that that at least builds. -- Flyhigh (talk) 11:42, 6 June 2015 (EDT)

perhaps the segfaults are related to using 32-bit cinaries with the new WS/2.0 scenery?? just an idea -- Hamzaalloush (talk) 12:33, 6 June 2015 (EDT)

is there any way to share the broken stuff so that we can take a look - i.e. how about committing broken stuff to a separate topic branch, so that people can more easily follow what is going on and help find a solution ? Alternatively, let's exchange the corresponing logs on the wiki. Obviously, it would help to know if any of the build/run-time issues are affected by type of build and/or the target/library built. My suggestion would be to start with self-contained tests/demos unrelaeted to FG, and maybe switch to running FG/SG unit tests afterwards ?--Hooray (talk) 11:49, 6 June 2015 (EDT)

I'm now on i686-w64-mingw32.static, trying to build terragear all of a sudden. got some commits up on a new master-oldgcc-terragear branch containing patches for geos(fix exported symbols), boost(force static), and a terragear.mk with mingw specific patches that makes it compile 90% of the way. for last status updates overall, Flyhigh seems to interact pretty well the MXE community and got some changes commited to upstream[8][9]. i agree with the compile log's idea, let us make another topic for that, as long as we keep them even if solved for later reference. -- Hamzaalloush (talk) 12:21, 6 June 2015 (EDT)

Thank you for trying to get useful things merged upstream - regarding those crashes, if you can, please share your gdb backtraces using either the wiki or some form of pastebin. If that is not an option, I would suggest to prioritize working out a useful way to add better debugging support for those binaries (e.g. using Breakpad) so that we don't need to use gdb for getting a backtrace. Regarding PortableXDR: where is that being used currently, would it make sense to reuse the tinyXDR implementation in SG/FG (mp) or at least look at it for reference/patching ? If you think those segfaults may be scenery related, I would suggest to consider using the minimal startup profile to check that theory, i.e. by excluding scenery from being loaded altogether. Trying to build TG seems like another useful test case, so maybe we should add this to the roadmap, because it will pull in tons of other SG/FG dependencies. For troubleshooting segfaults, I would suggest to focus first on simple binaries, e.g. unit tests in SimGear and/or $FG_SRC/utils (e.g. stuff like fgviewer, fgjs etc). Maybe we could update the wiki to present a more up to date list of recent developments and/or required work that lies ahead of us ?--Hooray (talk) 16:33, 6 June 2015 (EDT)

fgfs backtraces

I'll work on getting the backtraces, but if I recall correctly, they didn't seem too helpful. PortableXDR is used in compiling hdf4; I don't think any of the other downstream libraries (gdal, netcdf, openscenegraph, simgear, flightgear) use it directly. Regarding the segfaults, they're actually happening at the end of "initializing subsystems" and the start of "finalizing subsystems". Also, running fgjs seems to work, as the "joystick" on my laptop (the hard drive) is detected, and fgviewer launches, but the earth is just a gray sphere. -- Flyhigh (talk) 00:05, 8 June 2015 (EDT)

is that also happening with the minimal startup profile being used ? and what about explicitly disabling scenery altogether ? If in doubt, please use the minimal startup profile and post your fgfs.log file so that we can take a look. It may be worthwhile to add a few SG_LOG() statements to the startup sequence in $FG_SRC/Main/fg_init.cxx to see what is going on - if that still isn't conclusive, I'd suggest to link in a debugging library like Breakpad which should be helpful to come up with a meaningful backtrace. Regarding fgviewer, what about the osgviewer/osgearthviewer binaries, do they work as expected ? I'd suggest that we add a table to the main article with all programs that are known to work/not work, so that we can more easily troubleshoot things in the time to come..

—This unsigned comment was added by Hooray (Talk | contribs) 10:02, 8 June 2015‎ (UTC)

I've uploaded the fgfs.log file, the command I ran, and the wine memory dump to a Github Gist. As for fgviewer, after including the path to the data directory, fgviewer works correctly. -- Flyhigh (talk) 11:38, 9 June 2015 (EDT)

I would suggest running this inside gdb to get an annotated backtrace with source code shown - alternatively, we need to prioritize adding support for BreakPad, which should help with this. Also, is this using the minimal startup profile or just the default settings at ksfo ? If in doubt, I would also check that all sg/fg unit tests are working correctly, to see if that brings up any suspects. But for troubleshooting purposes, I would definitely first try the minimal startup profile as per the article (no multiplayer, scenery, terrasync etc)--Hooray (talk) 14:14, 9 June 2015 (EDT)

I'v just read that link you posted about BreakPad, i like the idea of using a crash collecter to store the internal symbols of objects during compilation, so that it can be referenced against a mini-dump. it also support symbols generated in "OS X/Linux, DWARF" format, which is optimal for GCC. should we start to have a central server for crash collection? -- Hamzaalloush (talk) 17:41, 11 June 2015 (EDT)

OSG static build

I'm unable to build a static OpenSceneGraph. When building the present3D application, the linking stage fails, because it wants all of the static libraries used (directly and indirectly), not just the directly used ones. I've uploaded the ending of the build log into a gist. Any suggestions? -- Flyhigh (talk) 23:52, 7 June 2015 (EDT)

get into the CMake lists for that object that's failing to link, "present3D.exe". and define the missing libraries in "link_to_target". you would have to search google from which API these calling functions came from. -- Hamzaalloush (talk) 05:11, 8 June 2015 (EDT)

I did that for one round, but that yielded another set of undefined references, as those libraries called other libraries. Is there any way to compile it other than completely specifying all dependencies? -- Flyhigh (talk) 11:13, 8 June 2015 (EDT)

you could try to remove that target that's pulling tons of dependencies. if non of the other targets links with, maybe try to delete/edit its CMakeLists? alternatively you can disable these dependencies from the library. if these are GDAL related dependancies, then you can try to use a helper executable, like 'gdal-config.exe --dep-libs' and append these results into a variable, which is what i did for Terragear. https://github.com/hamzaalloush/mxe-clone/blob/master-oldgcc-terragear/src/cmake/FindGDAL.cmake (Line 27, find dependencies. Line 52, append to variable) -- Hamzaalloush (talk) 13:14, 8 June 2015 (EDT)

excessive qoutes, user talk style

i suggest removing some of the qouting, i'd offer remove my qouting if no one minds, as i look to make our page cleaner, also what about some cleaner talk/reply style, any ideas? as excessive threading makes the talk unreadable in some instances -- Hamzaalloush (talk) 20:12, 8 June 2015 (EDT)

sure, feel free to consider this "your" article and just go ahead - quotes are usually just intended to help "bootstrapping" new articles/projects, and should be phased out over time, unless some decisions/priorities need to be backed up by links to the corresponding statements - right now, the article has seen 1.8k views and dozens of edits, and it seems pretty safe to say that the article will stay around even if this effort should get stalled at some point, so unlike many other articles, this is already useful and documents and actual project.--Hooray (talk) 04:56, 9 June 2015 (EDT)

repository relevant talk

a section, that contains a list will be created that you'll have to fill up to reflect what your working on. elements in the list are: owner, what it does/what feature it is supposed to create, which architichture, linking type, clone command + url. also, i suggest we rebase each feature from master-oldgcc for now(until we get a working GCC 5). -- Hamzaalloush (talk) 17:58, 11 June 2015 (EDT)

Simgear Test Crashes

I've been under the assumption that the tests in Simgear are run automatically with every build. I didn't realize that thy had to be run manually (side-effect of using dh in Debian packaging, which does everything automatically for you). After running the tests, two tests crash, two tests go up to 100% CPU for at least a minute (before I killed those tests), one seemed to hang after completing successfully, and one seemed to have been called incorrectly (it was missing an argument). Below are the results:

• test-simgear_canvas_layout-canvas_layout.exe - Crash
• sg_pkgutil.exe - Crash, infinite recursion in what seems to be the Wine loader
• test_sock.exe - Hang after completion
• http_get.exe - 100% CPU
• http_svn.exe - 100% CPU
• decode_binobj.exe - Incorrect usage? Called with no arguments, when one argument was expected?

This was on x86_64-w64-mingw32.shared. I'll be compiling GDB soon, and hopefully, I can get a backtrace. -- Flyhigh (talk) 15:40, 13 June 2015 (EDT)

Here's the stacktrace for test-simgear_canvas_layout-canvas_layout crashing. Don't know how useful it is... -- Flyhigh (talk) 22:28, 16 June 2015 (EDT)

Not sure if this will help, but i notice your using the "release" runtime of boost rather than debug, so if you want to make sure, you can try rebuild boost with "./b2 ... variant=debug ...", atleast it will be helpful to look at the boost code as well. -- Hamzaalloush (talk) 17:37, 17 June 2015 (EDT)

Those functions are Boost unit testing framework which should just be calling the Simgear tests, so I don't think having a debug version of that library will help much. -- Flyhigh (talk) 11:43, 19 June 2015 (EDT)

can you tell us what steps can we take to compile Simgear and replicate these tests?(which branch, debugging methods etc, helpful if you wrote a section for debugging in the article). also, i was more concerned with the symbols included in the boost library rather than just the functions, for debugging reason. because i recieved info from Hooray that it is a suspected boost fault. -- Hamzaalloush (talk) 14:40, 19 June 2015 (EDT)

Regarding the building, it should be the current version of my master-oldgcc branch (c139f66) and building simgear and gdb for the x86_64-w64-mingw32.shared target. Note that the tests are not run, since it seems CMake decides to directly mark them as failed. For the testing itself, I copied all of the DLL files in usr/x86_64-w64-mingw32.shared/bin/, the gdb binary, and the test executables from the build (make sure you get these files before the build completes, as once the build completes, the build directory is removed; if it's too quick, you can induce a build failure by uncommenting the tests line, while will save the build directory) into Windows and ran gdb from there. (Note that you may need to copy libdl.dll to libdl.dll.a if you can't launch gdb) The reason I didn't run it from Wine in Linux is because the gdb prompt didn't seem to work correctly in Wine. -- Flyhigh (talk) 23:34, 19 June 2015 (EDT)

It's definitely something in Simgear's ghost (or in calling ghost). I've built a 64-bit shared version of Flightgear, and it crashes when calling nasal::internal::GhostMetadata::addDerived. GDB backtrace is here -- Flyhigh (talk) 11:05, 24 June 2015 (EDT)

That's Nasal/CppBind (Nasal related code using boost) - can you check if the Nasal unit tests are working correctly or not ? Also, what about Nasal standalone (no boost there). Alternatively, use boost to print out some more info: http://www.boost.org/doc/libs/1_55_0/libs/test/doc/html/utf/user-guide/runtime-config/reference.html Basically, I would suggest to eliminate fgfs to get down to this, some of the unit tests should be sufficiently self-contained to troubleshoot this: http://sourceforge.net/p/flightgear/simgear/ci/next/tree/simgear/nasal/cppbind/test/ --Hooray (talk) 11:20, 24 June 2015 (EDT)

Breakpad.mk

1. I'v been working towards getting Breakpad cross-compiled through MXE, and so far i'v been able to build using simple Mingw-specific portability fixes: [10]

i'm still having issues compiling the basic example provided in the LinuxStarterGuide

The breakpad.mk is a simple one, adopting autotools, using the $(MXE_CONFIGURE_OPTS) placeholder, and linking the ws2_32 runtime for <winsocks.h> support. right now, i have not found a mirror providing a recent Breakpad archive, so i tarred the source straight from a cloned repository, and are experimenting using a local server, feel free to modify the .mk file accordingly. breakpad_WIP.mk

the issues that i'm having consists of configure tests not running properly(suspected configure output variables: ANDROID_HOST_TRUE='#'), and therefore are applying headers that are android specific, and are expecting android specific runtime's. resulting in this error from missing Android SDK Issue 625: compile failed on android.

A configure patch is due, here's the full log of configure tests. -- Hamzaalloush (talk) 05:38, 22 June 2015 (EDT)

2. looks like the LinuxStarterGuide provided example, requires include of the vanilla source as well as linking with the built library libbreakpad_client.a("First, configure your build process to link libbreakpad_client.a into your binary, and set your include paths to include the src directory in the google-breakpad source tree."), but the source unfortunately requests the Android SDK through headers such as elf.h/link.h/ucontext.h. therefore for all intents and purposes, the binaries built from Breakpad's cross-compilation are working, although not tested with real application, i could get an output on the terminal when invoking them, what's left is integration and some form of testing which i am unable to do. -- Hamzaalloush (talk) 17:04, 22 June 2015 (EDT)

Python module for mxe

See FGPythonSys for the main article about this subject.

Depending on how this turns out, we may need to add a python.mk module so that host applications with embedded Python support can be built.--Hooray (talk) 06:20, 28 January 2016 (EST)