Update and cleanup OS X libraries #4362
Labels
No Label
Closed
Duplicate
Closed
Fixed
Closed
Invalid
Closed
Needs info
Closed
Won't fix
Closed
Works for me
Difficulty
Hard
Difficulty
Medium
Difficulty
Simple
Needs Info
Priority
1: Release Blocker
Priority
2: Must Have
Priority
3: Should Have
Priority
4: Nice To Have
Priority
5: If Time Permits
Theme
AI
Theme
Art & Animation
Theme
Atlas editor
Theme
Build & Packages
Theme
Core engine
Theme
Internationalization & Localization
Theme
Maps
Theme
Multiplayer Lobby
Theme
Music & Sound FX
Theme
Network
Theme
Non-game systems
Theme
Simulation
Theme
UI – Game setup
Theme
UI – In-game
Theme
UI – Miscellaneous
Theme
UI & Simulation
Theme/Website
Forum
Type
Defect
Type
Enhancement
Type
Task
No Milestone
No project
No Assignees
8 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: wfg/0ad#4362
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This patch is to keep track of the need library updates for the current milestone. Updates are done in https://trac.wildfiregames.com/browser/ps/trunk/libraries/osx/build-osx-libs.sh
I created a branch that relies on Homebrew to download all non-bundled libs (except for gloox, as we do not use ssl and brew does).
https://github.com/wraitii/0ad/tree/OSX_libs_brew
This seems to compile on my system (10.11.6), and has the advantages of:
Now we voluntarily didn't use brew before. I'm not exactly sure why, but here are a few notes:
I don't see why we should/could not rely on it now, at least.
NB: I removed libiconv because it seemed to be only used to compile the others, but I'm not actually sure about that.
As a sidenote: using brew libraries and hot linking seems to work but fails to compile into a binary that's self-sufficient, it'll probably take more changes.
As a side-sidenote, wxwidgets 3.0.2 no longer compiles on macOs 10.12 (Sierra), you have to use 3.1.0
edit:hm actually not either, we'll have to port an upstream patch.
Can we push that to A23 or is there an important library update that needs to be performed before packaging?
Patch removed.
Most (all?) libraries in build-osx-libs.sh should be updated for a22, some due to security issues.
Wraitii use of Homebrew can wait for a future release.
Suggested for a22:
Alright, I hope wraitii can look into that.
I'm going to look into performing a few updates on Windows as well, they are long due.
Besides that zlib version not being available anymore (reported in #4639), nigel87 also uses OSX Sierra 10.12 which fails to build wxWidgets due to including quicktime which was dropped from that OS:
http://trac.wxwidgets.org/ticket/17639
https://forums.wxwidgets.org/viewtopic.php?t=42856
He tried passing
WXWIDGETS_VERSION="wxWidgets-3.0.3"and added--disable-qtkitforCONF_OPTSin L356, but that didn't help.See
12b23fc95fPhab:D679
See
e035359b73Four exploits in the latest release of libxml2 mentioned in https://code.wildfiregames.com/D679?id=2703#inline-12860 should be either patched or it should be confirmed that we are not affected.
Rest of the library versions should be checked for exploitable vulnerabilities too.
Phab:D699 for the libxml2 snapshot.
I set this to backlog because noone has the will to update libxml2 to the most recent dev snapshot before the alpha 22 release and because we will need a new ticket to keep track of updates for the next releases if this was closed as fixed.
for macosx, im encountering
error: 'connectx' is only available on macOS 10.11 or newerwhile installing libraries on curl-7.54.0,fix is to update to curl-7.56.0, see discussion here: https://github.com/VCVRack/Rack/pull/200
quick fix is to update this line to :
CURL_VERSION="curl-7.56.0"in libraries/osx/build-osx-libs.shI am going to perform the curl change on macOS ASAP.
See
fce1207e4dGiven #4790 was recently closed, here is an update of latest libraries and current status (to be A23) for OS X:
Most are a bit behind and some are possibly security related. Patching is trivial, but it would likely need some testing.
Will update some low risk libraries here.
See
db40a0753eGiven #4790 was recently closed, here is an update of latest libraries and current status (to be A23) for OS X:
Some are still a bit behind and possibly security related. Patching is trivial, but it would likely need some testing.
See
3759e60a44@stanislas69 , @trompetin17
If you are testing OS X you may want to have a look at this. These two are still open:
Hey Fabio, I think boost is still being used, I remember having issues with when I tried to switch to the VS2017 compiler.
Shouldn't we use the latest libpng possible ?
I meant just "boost system" should no longer be needed. On Linux it is no longer used. So I mean change:
--with-libraries=filesystem,system
to just:
--with-libraries=filesystem
And I would agree to update most/all libraries to their latest version, especially the ones with security implications.
It should be tested by someone having OS X.
Patch set to: Phab:D1691
On a related note, it wouldn't be a bad idea to do checksum verification for all downloaded files within the script (especially since this is used for official releases), but that might belong in a separate ticket.
Replying to wraitii:
The reason for the custom build script is mostly related to bundle distribution, where we want to target a specific SDK and minimal API version. If there's a package manager that can do that reliably, and not pick up incompatible libraries built against other SDKs or the local system libs, then we could certainly switch to it.
Or maybe have some sort of isolated package manager install just for the 0 A.D. build, but IMO the point is we can't just take any old packages built in other contexts and expect them to work in the bundles. And using static libs was 100% reliable, whereas picking up dylibs was kinda flaky back then (lots of major changes between OS X versions, plus Apple tended to not update their 3rd party libs).
What we don't want is a dev that already uses Homebrew for building random stuff, and then builds a 0 A.D. bundle using a Homebrew-based solution, and then it doesn't work on other macOS versions. That is what used to happen, both with Homebrew and MacPorts.
Another option would be to use e.g. Homebrew for non-release builds and use this script for bundle releases. But I thought having a single build path per OS was most sensible, even if the first full build does take a while (I forget how long it took, maybe 15-20 minutes back in 2015? on a quad-core 8GB RAM VM)
FYI (This comment is 2 years old) - my current opinion is that x agree with you.
Still it's convenient for some libraries (wxwidgets notably)
Replying to wraitii:
I figured, but couldn't remember if we ever discussed that or where, and wanted the original intent documented for posterity :)
I think looking back, my regret would be choosing bash instead of Python, not only because we might be able to reuse existing Python-based projects for this sort of thing (I've used at least one on a different project), but also the possibility of building many of the same libraries on Windows, which is currently not automated at all (even more of a pain and much more time-consuming).
I think eventually we could have a prebuilt package of macOS libraries with whatever the earliest SDK/API we support is. Similar to what we do with Windows now (in SVN), that way almost nobody would ever need to run this script at all. Just download the package and you're ready to build 0 A.D.
In
59ee761808by historic_bruno:Fixes GnuTLS build on macOS.\
Fixes macOS linker warning "PIE disabled absolute - addressing not allowed".\
\
Updates nettle to 3.5.1, GnuTLS to 3.6.8, gloox to 1.0.22.\
Disables TCP fast open feature of GnuTLS (requires 10.11, no SDK build support).\
Fixes GnuTLS detection of GMP by adding it to LIBS flag.\
Disables getaddrinfo on gloox 1.0.22. Lobby connections failed during server hostname resolution.\
Adds --with-pic to GMP build to force consistent PIC usage.\
Adds -N flag to patch commands to avoid reapplying them.\
Removes unneeded build flags.\
Documents --enable-fat configure flag: GMP and nettle detect CPU-specific features, fat binaries let us build and run them on different CPUs (see D1772).\
\
Fixes #5453, #5489. Refs #5481.\
Tested by: kali0ad, trompetin17\
Reviewed by: trompetin17\
\
Differential Revision: https://code.wildfiregames.com/D2057\
Note to self: replace http URLs with https.
466aca208aupdated libsodium to 1.0.18.Patch removed.
There's a post in the forums with a few ideas for lib upgrades.
https://wildfiregames.com/forum/index.php?/topic/28059-svn-public-alpha-24-version-on-macos-ui-problem/&tab=comments#comment-395406
I'm not sure there's a use for having this generic ticket though. Libs should be updated either because there is a problem of some sorts, of because we want to keep it closer with the versions used by other platforms (e.g. win/linux).
I suppose we could add a point to some sort of release checklist to see if there are patch releases we haven't applied and consider applying them, and if there are major upstream releases to file a ticket to look into whether that's useful etc.
I'd suggest closing this but open to comments from others :)
More possible lib updates by @Stan:
See
eee35195e3Pushing to A25 as most updates have been performed already.
See
f6f4d6fe7aPush back