GNOME Archives Integration and FreeBSD Porting ============================================== :author: 藍ĉŒşç‘‹ (Ting-Wei Lan) https://wiki.gnome.org/TingweiLan/ :email: lantw@src.gnome.org :backend: slidy :icons: ++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++ *Archive integration* is very useful because we usually download and upload archives in order to transfer folders over the Internet. By providing simple functions and GTK+ widgets, developers can easily integrate archives support into their applications. - https://wiki.gnome.org/Outreach/SummerOfCode/2013/Projects/TingWeiLan_GnomeArchives[GSoC 2013 Project] - https://wiki.gnome.org/Design/OS/Archives[Design] and https://wiki.gnome.org/Design/Whiteboards/Archives[Whiteboard] *FreeBSD porting* is a work not only to let GNOME build and work out-of-the-box on FreeBSD but also to keep GNOME portable across different operating systems and compilers. GNOME version on *BSD will be more up-to-date, and package maintainers can be notified about broken GNOME modules right after the commit. - https://wiki.gnome.org/Projects/Jhbuild/FreeBSD[JHBuild on FreeBSD] - http://www.marcuscom.com:8080/viewvc/viewvc.cgi/marcuscom[MarcusCom ports] This Slide ---------- Download this slide and its source code here: - http://w.csie.org/~b01902062/gnome[] - http://f1.csie.org/~lantw44/gnome[] - http://s1.csie.org/~lantw44/gnome[] - http://p1.csie.org/~lantw44/gnome[] [NOTE] These download links may not be always alive. Please download them now. [TIP] There are many links included in this slide. Some links may not be shown during this talk. If you are interested in them, please download this slide and view it on your device. Archives Integration -------------------- [role="incremental"] - It was a *Google Summer of Code* project in 2013. - Users don't have to know what is an archive. It just work like a folder. * *Default values* should work. - Users can create and extract archives automatically when needed. * No need to use *file-roller* or *command line* to create an archive. * No need to specify *a lot of options*. - Advanced users can also get their work done. * Users can disable the integration. * Users can specify which files should be *recognized as archives* and whether all files should be extracted. New library - gnome-autoar -------------------------- - A tiny library which uses *libarchive* and *GIO* as the backend to do work related to archives for regular applications. - Download it here: https://git.gnome.org/gnome-autoar - There is currently no stable releases, but I think the initial (possibly experimental) release will be made if my archives integration patch for https://bugzilla.gnome.org/show_bug.cgi?id=707647[Evolution] is accecpted. - API may be changed when needed. New functions and widgets may be added if some applications require them. gnome-autoar Functions and Goals -------------------------------- [role="incremental"] - *Make archives integration simple for most GLib/GTK+ applications* * Specify source and destinations files or directories and connect callbacks, and gnome-autoar will do all other things for you will do all others. * It should not cause portability or stability problems in applications. - *Integrated with GNOME desktop* * Retrieves user archives settings using GSettings. * Provides GObject-based C API and bindings using GObject introspection. * Provides common GTK+ widgets to select preferred archive formats. - *Solve annoying problems* * Useless files can be excluded when extracting. gnome-autoar Non-goals ---------------------- [role="incremental"] - *gnome-autoar* does not replace *file-roller* or other advanced archives-processing tools * gnome-autoar currently only support libarchive as the backend, and there is no plan to add another backends until it is really needed. * It cannot handle password-protected archives. - *gnome-autoar* is not a GLib binding for libarchive * It does not provide any direct binding to libarchive. Using gnome-autoar ------------------ . Retrieve user settings using `autoar_pref_new_with_gsettings()` . Create an object to do the work using `autoar_create_new()` or `autoar_extract_new()` . Connect callbacks to signals you are interested in, such as `progress` and `completed` . Start the work using `autoar_create_start_async()` or `autoar_extract_start_async()` Automatically create archives ----------------------------- - `gnome-autoar/autoar-create.c` - Automatically decide the output archive name based on the name of source files and directories - Automatically add a top-level directory in the archive if multiple sources are selected - Resolve hard links using functions provided by libarchive Automatically extract archives ------------------------------ - `gnome-autoar/autoar-extract.c` - Always produce single file or single top-level directory in the output directory - The name of the output file or directory is the same as the source archive without extension, so users can easily know where the extracted files come from - Ignore unneeded files such as `__MACOSX` - Modify bad path names if it will cause extracted files to be located outside the single top-level directory Epiphany with Archives Integration (1) -------------------------------------- - Patches available for Epiphany 3.10, but it currently does not work with the master branch because of conflicts. - Extracting archives: * Archives support can be enabled in the preferences dialog. * Downloaded archives will be automatically extracted. - Screenshots * image:images/Epiphany-Downloading.png[] * image:images/Epiphany-Extracting.png[] * image:images/Epiphany-Extraction-Finished.png[] Epiphany with Archives Integration (2) -------------------------------------- - Creating archives: * Single file selection: ** An archive will be created if folders or multiple files are selected. * Multiple file selection: ** An archive will be created for each selected folder. * User should wait for the archive operation to complete or cancel the operation before select the next file ** A `WebKitWebView` can only have one `WebKitFileChooserRequest`. Epiphany with Archives Integration (3) -------------------------------------- image:images/Epiphany-Upload-Dialog.png[] Evolution with Archives Integration (1) --------------------------------------- - Latest patches can be found on https://bugzilla.gnome.org/show_bug.cgi?id=707647[GNOME Bug 707647]. - Mailing list post: * https://mail.gnome.org/archives/evolution-hackers/2013-August/msg00023.html - Extracting archives: * Autimatically check or uncheck the `Extract files from the attachment if it an archive' checkbox based on the file name extension and MIME type. * There is no progress bar currently. Evolution with Archives Integration (2) --------------------------------------- image:images/Evolution-Attachment-Save-Dialog.png[] Evolution with Archives Integration (3) --------------------------------------- - Creating archives: * image:images/Evolution-Attachment-Load-Creating.png[] * image:images/Evolution-Attachment-Load-Creation-Finished.png[] Empathy with Archives Integration (1) ------------------------------------- - Patches only tested on 3.10. It contains some bad hacks, so I have not notified the developers. - Tested using People Nearby. I did not know how to setup other accounts for file transfer. Empathy with Archives Integration (2) ------------------------------------- - Screenshots * image:images/Empathy-Creating-Progress.png[] * image:images/Empathy-Extracting-Archives.png[] Common problems --------------- When users need to create archives in order to transfer folders, temporary archives will be created under temporary directories used by the applications. However, files in temporary directories may not be removed after applications exit, which causes possbily wasted disk space. Archive formats selecting menu ------------------------------ - Used to select common archive formats - Created using `autoar_gtk_chooser_simple_new` - Screenshots * image:images/Autoar-FormatFilter-Simple.png[] Archive formats selecting dialog -------------------------------- - Used to choose all possible combination of formats and filters - Created using `autoar_gtk_chooser_advanced_new` - Screenshots * image:images/Autoar-FormatFilter-Advanced.png[] Possible problems of archive formats selecting dialog ----------------------------------------------------- - It shows *all formats* supported by libarchive. Users may be confused with this large number of options. * Users are expected to use the menu instead, but items in the menu is currently hard-coded. We may change it to GSettings, so users can configure their menus based on their preferences. - Users who do not know libarchive may not understand what is the difference between some similar formats: * *4 Tars, 2 CPIOs, 2 ARs.* * Documented in `libarchive-formats(5)` and `archive.h`. * We may hide these similar options by default, and add a checkbox called `show all options' to show all these confusing options. gnome-autoar API ---------------- [role="incremental"] - C API is almost [green]#done# * Its `_async` is different from the convention of GIO. I will change it to GIO-style and drop the `completed` signal if I have time. * We may make use of more `GTask` features. - GObject-introspection binding is [olive]#not very good# * Non-GObject types and macros become strange in GObject-introspection based bindings. * Some functions rely on converting enum into integer, which may not work on other languages. - [red]#No vala binding# gnome-autoar archive format workaround -------------------------------------- [role="incremental"] - Ideally, gnome-autoar should not behave differently when users choose different archive formats, as libarchive itself is an abstraction layer over these formats. - However, gnome-autoar still have to do different things depends on the chosen format because libarchive may return error if unexpected entry or data is encountered. * Search for `ARCHIVE_FORMAT_` in `gnome-autoar/autoar-create.c`. * libarchive may fail with no error code or message! It is hard to debug without reading libarchive source code. * This difference does not show in the archive format selecting dialog. This is bad because some format does not support directories, and users should be warned. gnome-autoar signal emission hacks ---------------------------------- [role="incremental"] - gnome-autoar does not use async I/O functions from GIO. Instead, it creates a new thread to do all work in it if async version of functions are requested. - Signals directly emitted in the working thread can cause problem because GTK+ widgets can only be manipulated in the main thread. - gnome-autoar currently use an ugly hack: * `gnome-autoar/autoar-private.c` * Manually pick arguments from the `va_list` and pack arguments into a struct. * Emit the signal in the main thread using `g_main_context_invoke`. * It not only causes signal delay but also does a lot of extra work in order to emit a signal. * Is there any *better way* to solve this problem without modifying all existing code? gnome-autoar Security (1) ------------------------- [role="incremental"] - Automatical extraction means users cannot review the content of the archives before extracting. - We need to prevent malicious archives from overwriting files or making mess in other directories. * Stop all work as soon as an error is found. * Leading `/' in path name is removed. * Remove `..' components if it caused extracted files to be located outside the destination. gnome-autoar Security (2) ------------------------- - Other protection is also needed: * Use `g_file_create` so existing files are not overwritten. * Check for symbolic links in the path names. A symbolic link may have leading `/' or `..' which cannot be found using the current check. - See how others do it: * Read related code from *file-roller*, *bsdtar*, or *GNU tar*. gnome-autoar Security (3) ------------------------- - Example bad archive --------------------------------------------------------------------------- lrwxrwxrwx 0 test users 0 May 3 23:38 rs/.git/refs -> ../../../.repo/projects/frameworks/rs.git/refs drwxr-xr-x 0 test users 0 May 3 23:27 rs/.git/logs/refs/remotes/ drwxr-xr-x 0 test users 0 May 3 23:27 rs/.git/refs/tags/ -rw-r--r-- 0 test users 41 May 3 23:27 rs/.git/refs/tags/android-4.4_r1 --------------------------------------------------------------------------- gnome-autoar Stability ---------------------- - The API change may cause crash, but it is not a problem once the API become stable. - Tests need to be written. Currently programs in the `tests` directory can only be used for developing, not testing. - It must be stable because it will be integrated into *core GNOME applications*, which must not crash. - I run *valgrind* and *clang static analyzer* on it from time to time to find possible bugs. gnome-autoar Portibility ------------------------ - gnome-autoar does not use any GCC extension. - gnome-autoar depends on GLib and GIO to do most work, which already solve many portibility problems. - All use of non-portable functions, including POSIX functions, will be disabled if it cannot be found using configure script. - Current master branch of gnome-autoar can be built unmodified on *GNU/Linux* (GCC, Clang), *FreeBSD* (Clang), *Windows* (MinGW-w64 GCC), and possibly other operating systems. gnome-autoar Unfinished Functions --------------------------------- - ACL support exists in libarchive, but ACL is currently not supported in gnome-autoar. I will add ACL support for Linux and FreeBSD when other required functions are completed. - Settings dialog is needed for users to edit their preferences. Users are not expected to modify settings using gsettings or dconf directly. [red]#High priority functions# FreeBSD Porting --------------- - GNOME *used to* be available on many different UNIX-like operating systems. - However, after *GNOME 3* is released, few non-Linux system have GNOME 3 packages, and no non-Linux system can run GNOME 3 without patches. - What is the problem? [role="incremental"] * Is there any big changes in GNOME 3 which causes porting becoming *very hard* or *impossible*? * Does GNOME 3 really *only* work on *Linux*? * Can we change the situation? FreeBSD ports and packages -------------------------- Search result of desktop environement packages using `pkg search` or `make quichsearch` in FreeBSD ports: [grid="none",cols="3,5,2,10",options="header"] |=========================================================== | Desktop | Ports | Version | | Xfce | x11-wm/xfce4 | 4.10 | [green]#Latest version# | KDE | x11/kde4 | 4.12.5 | [green]#Released less than one month ago# | Enlightenment | x11-wm/enlightenment | 0.17.5 | [green]#Released half a year ago# | GNOME | x11/gnome2 | 2.32.1 | [red]#Released in 2010 (no longer maintained by upstream)# |=========================================================== GNOME packages in FreeBSD is outdated! FreeBSD GNOME Porject --------------------- - http://www.freebsd.org/gnome/[] - Information on this page is also outdated - It is not easy to find GNOME 3 porting status on this page MarcusCom ports --------------- - http://www.marcuscom.com:8080/viewvc/viewvc.cgi/marcuscom/[] - http://www.marcuscom.com/downloads/marcusmerge[] - This is where GNOME ports in FreeBSD prepared - You will need `marcusmerge` script to integrate it into your ports - MarcusCom ports is continuously updated, but no big changes are made into official FreeBSD ports except for GLib, GTK+ and other fundemental libraries. Before this project - Using the MarcusCom ports ----------------------------------------------- [role="incremental"] - In late 2012 or early 2013 * It was a mix of *GNOME 3.4 and 3.6* * A lot of *conflict packages and broken dependencies* * Failed to install some core packages (e-d-s) and broke other desktop (Xfce) - In August, 2013 * It was *GNOME 3.6* * *GDM* did not start * *GNOME Shell* did not show the top bar or panel - It was *hardly usable*! Before this project - Building GNOME using JHBuild -------------------------------------------------- [role="incremental"] - In Augest, 2013 * JHBuild could build almost *everything in GNOME* on Linux. Did it work on FreeBSD? [role="incremental"] ** [red]#Answer: No. It stopped in very early stage.# ** [red]#Some developers are not interested in FreeBSD support.# - In November, 2013 * Did JHBuild work on FreeBSD now? [role="incremental"] ** [red]#Answer: No. It was even more broken.# - *How many problems* have to be fixed to make JHBuild work on FreeBSD? [role="incremental"] * [green]#More then 150!# (Counted on GNOME 3.12 releasing day) * New problems will be found and we need to continuously fix them. 2013 status of GNOME on FreeBSD ------------------------------- - It did not work at all - No GDM, GNOME Shell, GNOME Online Accounts, GNOME Control Center, GNOME System Monitor, Epiphany, Evolution ... 2014 status of GNOME on FreeBSD ------------------------------- [role="incremental"] - GNOME 3.12 on FreeBSD VM image was available on GNOME 3.12 releasing day. * Blog post of desrt: http://blogs.gnome.org/desrt/2014/03/26/gnome-3-12-and-freebsd-and-a-virtual-machine/[] * Download link: https://download.gnome.org/misc/promo-vm/[] - Nearly all modules can be successfully built using JHBuild if you follow the instruction on the project wiki page. * JHBuild on FreeBSD status and instruction: https://wiki.gnome.org/Projects/Jhbuild/FreeBSD[] - FreeBSD GNOME ports maintainers make packages much faster than before. * Many GNOME 3.12.2 packages are available in MarcusCom ports now. Who work on this project? ------------------------- - *desrt* - Ryan Lortie - GNOME developer - *kwm* - Koop Mast - FreeBSD ports committer - *lantw44* - Ting-Wei Lan - *gusi* - Gustau Perez - FreeBSD ports committer - Thank for *[olive]#OpenBSD# ports committers* who solved problems on BSD before this project was started - Thank for all other people who help us solving bugs! Here is the problem summary --------------------------- [role="incremental"] - Both fixed and unresolved problems are listed * Knowing *fixed problems* helps prevent new problems * Knowing *unresolved problems* interests everyone to fix them - Not all problems are GNOME problems * *FreeBSD* itself also has problems to solve * *Libtool* is another source of annoying problems * *WebKit* has many problems which are *very hard* to find and solve FreeBSD Problem - KMS drivers ----------------------------- pass:[FreeBSD] [role="incremental"] - KMS support is incomplete in FreeBSD 9.2 and 10.0 * *Only one Xorg* instance can be started. The second Xorg does not show anything, and it may cause the system to crash. * After starting Xorg using KMS drivers, text console are no longer usable. You must have another computer which you can *ssh* into the machine if your desktop crashes. ** [red]#Both are big problems when testing or developor for an unstable desktop.# * *Mesa* and *Intel drivers* are too old. - Some problems may be fixed in FreeBSD 9.3 and 10.1 * Xorg problems is tracked in the https://wiki.freebsd.org/Graphics[Graphics] page. * https://wiki.freebsd.org/Newcons[Newcons] will solve the text console problems. FreeBSD Problem - .pc files missing ----------------------------------- pass:[FreeBSD] - Most libraries in the base system, such as *openssl*, *libarchive*, and *zlib*, do not have their .pc (pkg-config) file installed. * This causes `configure` script and `jhbuild sanitycheck` fail to detect the installed library. * Some modules such as https://bugs.webkit.org/show_bug.cgi?id=127061[*WebKit*], https://github.com/libgit2/libgit2/issues/2118[*libgit2*], and https://git.gnome.org/browse/gnome-autoar/commit/?id=ace549c774b91f1a92f7d172f2dbf6641733b0e4[*gnome-autoar*] have been patched to workaround this problem. ** [red]#However, we do not expect all developers accept the workaround, as it is FreeBSD that causes the problem.# * The correct solution is to provide these .pc files in the base system or ports / packages. Libtool Problem - Libtool is broken on FreeBSD ---------------------------------------------- pass:[Libtool] [role="incremental"] - Libtool uses different version schema on FreeBSD * I don't know why this difference was needed. It causes *huge problems* for GNOME modules. * It causes the `SONAME` to be changed every minor update. - For example, you `jhbuild` the GLib yesterday, with `SONAME = libglib-2.0.so.4098`. * You run `jhbuild` again today, and 5 commits were added, which causes it to be changed to `SONAME = libglib-2.0.so.4100`. * All modules you built yesterday are *broken* because of the `SONAME` change! - Currently we modify the libtool to get the same version schema as on Linux. FreeBSD Problem - .la files installed ------------------------------------- pass:[FreeBSD] [role="incremental"] - .la files hard-code several flags, including library path. * This causes wrong version of libraries to be found. - For example, a GNOME module needing both libintl and GLib are being built in a `jhbuild` prefix. * `/usr/local/bin/libintl.la` causes `/usr/local/lib/libglib-2.0.so.0` to be pulled in. * This will cause undefined reference because GLib installed on the system is always older than the one installed by `jhbuild`, which causes new symbol cannot be found. - Most distributions automatically remove the installed .la files. * `jhbuild` also do the same thing. FreeBSD Problem - Wrong D-Bus machine-id file path -------------------------------------------------- pass:[FreeBSD] [role="incremental"] - D-Bus machine-id file should be located at `/var/lib/dbus/machine-id` * However, there is no `/var/lib` on FreeBSD and `/var/db` should be used instead. * Therefore, all packages contain this path are patched to use `/var/db/dbus/machine-id`. - We need to make the machine-id path to be a part of the D-Bus spec * FreeBSD ports maintainers should stop using this patched path. FreeBSD Problem - Namespace conflict ------------------------------------ pass:[FreeBSD] [role="incremental"] - `Error closing file: no error: 0` when running `g-ir-scanner` for libgtop * `g-ir-scanner` deletes generated C code and executable by default. ** Getting the program which causes this problem is not a trivial task. ** I used `LD_PRELOAD` to disable the `unlink` function to debug the problem, but the correct way to do it is to set an environment variable. - Both FreeBSD libgeom and GLib have a function called `g_close`. * Their return values have different meaning. * Until libgeom stop using the `g_` namespace, any GLib applications cannot use libgeom. * We https://bugzilla.gnome.org/show_bug.cgi?id=723685[ temporarily disable the use of libgeom] in libgtop to workaround it. // Command GNOME Problem - Hard-coded interpreter path ------------------------------------------- pass:[GNOME] - `#!/bin/bash` and `#!/usr/bin/python` are the most common * They should be changed to `#!/usr/bin/env bash` and `#!/usr/bin/env python`, respectively. * Alternatively, you can use `configure` and `#!@PYTHON@` to detect the path and generate the script with correct interpreter path. - It is very easy to fix, but it is annoying. GNOME Problem - Hard-coded command name --------------------------------------- pass:[GNOME] - Don't use `make` in Makefiles. Please use `$(MAKE)`. * We cannot expect `make` is always GNU make. * Almost all Makefiles in GNOME can only work with GNU make. GNOME Problem - "Bashisms" -------------------------- pass:[GNOME] [role="incremental"] - Avoid "Bashisms" in `configure` script if possible. * You can use the `checkbashisms` tool to check. - If bash-specific feature is really needed, please make sure the first line of the `configure` script is `#!/usr/bin/env bash`. * You can use `m4_divert_once([BINSH], [@%:@! /usr/bin/env bash])` to do it. (Copied from https://git.gnome.org/browse/aisleriot/commit/configure.ac?id=f2525b5fbea5d221a6a361a97334dfec956665dd[ an aisleriot commit]) GNOME Problem - Non-POSIX command line option --------------------------------------------- pass:[GNOME] - `sed -i` is the most common problem. - `cp -d` and `sed` with extended regular expression is also problems. * Please check the http://pubs.opengroup.org/onlinepubs/9699919799/[POSIX standard] before using commands. GNOME Problem - Behavior difference on non-Linux system ------------------------------------------------------- pass:[GNOME] - `sed: stdout: Bad file descriptor` * Hard to find the cause of this strange error because `truss` and `ktrace` do not give meaningful results. - It is cause by the behavior difference of `g-ir-scanner` on non-Linux system * `g-ir-scanner` does not use libtool on non-Linux system by default. * You can always add `--libtool=$(LIBTOOL)` to `g-ir-scanner` arguments, but using https://wiki.gnome.org/Projects/GObjectIntrospection/AutotoolsIntegration[ INTROSPECTION_MAKEFILE] is recommended. GNOME Problem - User variables are overrided in Makefiles --------------------------------------------------------- pass:[GNOME] - `CFLAGS`, `CPPFLAGS`, `LDFLAGS`, `CC`, `LINK`, ... are all user variables * There are always reasons for users to set these variables. * Makefiles must not ignore them if users already set them. - If your module break when these variables are set * Then you should fix your module, not override them in Makefiles. // Syntax GNOME Problem - Function return type mismatch --------------------------------------------- pass:[GNOME] [source,c] int my_object_run (MyObject* myobj, int do_something) { g_return_if_fail (MY_IS_OBJECT (myobj)); /* other code */ } - This is the [red]#most common# problems * Mismatched return types cause neither warning nor error for GCC, but it is an error for Clang. * We cannot find a suitable GCC argument which can be used to catch this bug. - Here is the [green]#fixed problem list# (11 bug reports) * https://bugzilla.gnome.org/show_bug.cgi?id=722520[bijiben] https://bugzilla.gnome.org/show_bug.cgi?id=726808[cogl] https://bugzilla.gnome.org/show_bug.cgi?id=722002[file-roller] https://bugzilla.gnome.org/show_bug.cgi?id=722335[folks] https://bugzilla.gnome.org/show_bug.cgi?id=722024[gdl] https://bugzilla.gnome.org/show_bug.cgi?id=727530[libgweather] https://bugzilla.gnome.org/show_bug.cgi?id=723847[libmediaart] https://bugzilla.gnome.org/show_bug.cgi?id=723351[totem] https://bugzilla.gnome.org/show_bug.cgi?id=722021[rygel] https://bugzilla.gnome.org/show_bug.cgi?id=722023[seahorse] https://bugzilla.gnome.org/show_bug.cgi?id=725728[seahorse] GNOME Problem - Wrong `main` function declaration ------------------------------------------------- pass:[GNOME] - `main` should be declard as `int main (int argc, char* argv[])` * Wrong declaration does not cause build problem for *GCC*, but it is an [red]#error# for *Clang*. - Here is the [green]#fixed problem list# (2 bug reports) * https://bugzilla.gnome.org/show_bug.cgi?id=722595[gnome-shell] https://bugzilla.gnome.org/show_bug.cgi?id=723065[gnome-maps] GNOME Problem - C99 `inline` functions -------------------------------------- pass:[GNOME] [source,c] inline int func (int arg) { return arg + 1234 /* some_result*/; } - C99 `inline` is different from C++ `inline` * The above code may causes undefined reference to `func` if optimization is disabled. * If you don't know what is the difference, please read the http://clang.llvm.org/compatibility.html#inline[compatibility note] on the Clang website. - By default, GCC use GNU89 mode, which causes `inline` to be the same as C++ * Always use `static inline` or `extern inline`. // Library GNOME Problem - Format string troubles -------------------------------------- pass:[GNOME] - `gnome-common` contains some compiler flags which catch format string error * This is useful because wrong format string usage can cause crash or security issue, but format string checking is stricter in *Clang* than in *GCC*, which causes build failure with *Clang*. - It is nice to fix them, but we do not have much time * Only core library, such as GLib and libsoup, have this problem solved. * We currently use `-Wno-error=format-nonliteral` to workaround it. - Here is the [red]#remaining problem list# * anjuta, brasero, evince, gedit, ghex, gitg, gnome-photos, gnome-terminal, gnumeric, goffice, libgxps, nautilus-sendto, vinagre, xchat-gnome GNOME Problem - Missing `#include` ---------------------------------- pass:[GNOME] - POSIX requires some headers to be included to use some functions * However, we usually do not check this requirement if our code can successfully built in our platform. - This is not necessarily the problems of GNOME developers * Sometimes the system headers are broken, but we still need to workaround them. GNOME Problem - Missing or non-existent libraries ------------------------------------------------- pass:[GNOME] - `-ldl` is required on GNU/Linux to use functions such as `dlopen`. * However, there is no `libdl` on FreeBSD. FreeBSD have these dynamic linking functions in `libc`. - There is a common patch which we use to fix the problem. * Replace all `-ldl` in `Makefile.am` with `$(LIBDL)`. * Add the following code to `configure.ac`: --------------------------------------------------------------------------- AC_CHECK_LIB(c, dlsym, LIBDL="", [AC_CHECK_LIB(dl, dlsym, LIBDL="-ldl")]) AC_SUBST(LIBDL) --------------------------------------------------------------------------- GNOME Problem - Non-POSIX `LC_*` and `_NL_*` constants ------------------------------------------------------ pass:[GNOME] - `_NL_ADDRESS_POSTAL_FMT`, `_NL_MEASUREMENT_MEASUREMENT`, `_NL_ADDRESS_LANG_TERM`, ... are non-standard * These constants are (possibly undocumented) glibc extensions. * Unconditionally use these constants cause build failure on non-glibc systems. - These non-standard are not macros, but enums * It cannot be fixed by adding `#ifdef` check. * Use `AC_CHECK_DECL` works. GNOME Problem - Non-POSIX C functions ------------------------------------- pass:[GNOME] - Some convenient and nice functions are only provided by glibc * Do not use them if GLib already provides alternatives or it can be easily replaced by other portable functions. - Here is the already [green]#fixed problem list# * gnome-terminal: `get_current_dir_name` * librsvg: `canonicalize_file_name` * zeitgeist: `get_nprocs_conf` // Kernel and system calls GNOME Problem - "Linuxisms" --------------------------- pass:[GNOME] - Some system calls are only available on Linux * `signalfd`: https://github.com/01org/dleyna-renderer/pull/142 - Some system calls behave differently * `shm_open`: https://bugs.webkit.org/show_bug.cgi?id=128572 * Do not expect `select` will tell you the remaining time. - Please check POSIX standard before using them. - http://www.freebsd.org/doc/en/books/porters-handbook/dads-avoiding-linuxisms.html[ Avoid Linuxisms] section in FreeBSD porters handbook. // X Window GNOME Problem - Bad X Window operations --------------------------------------- pass:[GNOME] * Your *Xorg* is probably too old - *Upgrade your Xorg* may fix the problem. - If your problem are fixed because of upgrading, there is *still possible* that there are bugs in Xorg or GNOME. * It takes time to find the real cause of the problem. - https://bugzilla.gnome.org/show_bug.cgi?id=724364 // Dependencies GNOME Problem - Hard-dependecy on udev -------------------------------------- pass:[GNOME] - Many software related to hardware management uses *udev* directly on Linux * There is no *udev* on FreeBSD. FreeBSD has *devd*, but their interface are different. * *libudev* developers reject portability patches. - *HAL* is deprecated, and there is currently no alternatives * GLib may grow an abstraction layer for this. * FreeBSD developers have tried to port some software from *HAL* to *devd*. Xorg currently have experimental *devd* port on FreeBSD. * FreeBSD developers write https://github.com/freebsd/libdevq[libdevq] to provide similar functions in order to port some software. GNOME Problem - Hard-dependecy on NetworkManager ------------------------------------------------ pass:[GNOME] - NetworkManager is currently only available on Linux * It requires headers from Linux. - It is possible to port NetworkManager to other systems * NetworkManager developers accept portability fixes. GNOME Problem - Hard-dependecy on systemd (1) --------------------------------------------- pass:[GNOME] - *Mutter* currently hard-depends on *wayland* and *systemd* * *Wayland* is currently not available on non-Linux systems, but it is possible to make it work and the upstream allow these portability patches. * *Systemd* is not portable to non-Linux systems, and the upstream do not accept any portability patches to non-Linux systems. * This is caused by https://git.gnome.org/browse/mutter/commit/?id=912cf8ee5f13eac66c361581919a4ffea9dd06cf[ the merge of wayland branch] in the last month * We do not have time to read and patch these tens thousands of lines of code, so both *mutter* and *GNOME Shell* 3.13 cannot be built. * Help us to fix https://bugzilla.gnome.org/show_bug.cgi?id=728185[GNOME Bug 728185]! GNOME Problem - Hard-dependecy on systemd (2) --------------------------------------------- - There will be more modules in GNOME using *systemd API* for extra features and enhanced user experience. * It is possible to implement some required systemd API on non-Linux systems. There is a http://www.freedesktop.org/wiki/Software/systemd/InterfacePortabilityAndStabilityChart/[ Portability chart of systemd API]. * We need software implementing systemd API in *BSD. * There is a http://www.google-melange.com/gsoc/project/details/google/gsoc2014/kremlin/5639274879778816[ Google Summer of Code Project] working on this! * Although http://www.openbsdfoundation.org/gsoc2014.html#systemd[ this is an OpenBSD project], we expect it to work on other BSDs. WebKit Problems (1) ------------------- pass:[WebKit] - Although WebKit is wildly used and it is an core dependency of GNOME, it is the most problematic modules. * More than *10 problems* have to be fixed in order to make it work on FreeBSD. * Its code size is *very large*, and it is hard for non-WebKit developers to find the cause of build failure and runtime bugs. ** *6 GiB* of disk space is required to checkout the source tree and complete the build. * It takes *more than one hour* to build WebKitGTK+ with my 2-core CPU. WebKit Problems (2) ------------------- pass:[WebKit] [role="incremental"] - All WebKitGTK+ programs shows a lot of `Failed to create shared memory file' error without showing any webpage. * Caused by invalid arguments in the `shm_open` system call. * Found by searching millions of lines of output of `truss` and `ktrace`. * https://bugs.webkit.org/show_bug.cgi?id=128572[WebKit Bug 128572] - LLINT and JIT are not enabled on FreeBSD * There is a lot of error message from the compiler. * It is easy to fix, but it is hard to find the cause. * https://bugs.webkit.org/show_bug.cgi?id=126502[WebKit Bug 126502] and https://bugs.webkit.org/show_bug.cgi?id=128898[WebKit Bug 128898] WebKit Problems (3) ------------------- pass:[WebKit] [role="incremental"] - `libtool: link: object name conflicts in archive: .libs/libjavascriptcoregtk-3.0.lax/libWTF.a ...` * The names of object files produced by autotools and libtool are already very long. Why there are conflicts? * Caused by the use of non-POSIX flags in `ar`, which causes names of object files to be truncated to 15 characters. * https://bugs.webkit.org/show_bug.cgi?id=128596[WebKit Bug 128596] - `eval: ar: Argument list too long` * I already use a large number for `sysctl kern.argmax`. Why it is still to long? * Caused by the `max_cmd_len` value of libtool. Change it to `max_cmd_len=250000` to fix the problem. Is GLib really cross-platform? ------------------------------ [role="incremental"] - GLib is a core library which should be portable * However, the first build failure in JHBuild was GLib in 2013. * FreeBSD ports contains more than 20 patches for GLib. - New policy of GLib * https://wiki.gnome.org/Projects/GLib/SupportedPlatforms * Setup daily builders to catch problems early. JHBuild Features - Condition Set -------------------------------- [role="incremental"] - `configure` arguments set in the moduleset files is used for GNU/Linux * We needs different arguments for FreeBSD. * Our `jhbuildrc` become filled with `module_autogenargs`. - *New JHBuild feature to solve the problem - condition set* * https://bugzilla.gnome.org/show_bug.cgi?id=720839[GNOME Bug 720839] Patches written by *desrt*. * Example: `` or `` * JHBuild works for many modules without `module_autogenargs` on FreeBSD. * We hopes the https://git.gnome.org/gtk-osx[gtk-osx] project can finally move their work into JHBuild itself and stop maintaining their fork. JHBuild Tinderboxes ------------------- [role="incremental"] - JHBuild can automatically build modules without user interaction and produce HTML reports. * It is used to be used by http://build.gnome.org, but the site uses GNOME-Continuous now. - We have setup two JHBuild Tinderboxes to catch problems early. * http://jhbuild.pkgdemon.com/tinderbox set up by *kwm*. * http://jhbuild.csie.org:9080/ set up by *lantw44* (myself). * We can now catch new problems in one day! Status of GNOME in MarcusCom ports ---------------------------------- [role="incremental"] - GNOME 3.12.2 almost works * Someone needs to test it. * Someone needs to write documentation. Upgrading from 2.32 to 3.12 is not an easy task. - Help FreeBSD ports maintainers test their new ports * Download the http://www.marcuscom.com/downloads/marcusmerge[marcusmerge] script and setup your http://fossil.etoilebsd.net/poudriere[poudriere] to build latest GNOME in a jail without breaking your system. * Join discussion in *#freebsd-gnome* on freenode. Help us make GNOME 3.14 work again ---------------------------------- - Currently the biggest problems are `mutter` and `gnome-shell` no longer work because of the hard dependency of `systemd` and `wayland` * Please provide patches for https://bugzilla.gnome.org/show_bug.cgi?id=728185[GNOME Bug 728185] * We have no time to fix it now. We need your help. * Ask for *jasper* in #gnome-hackers on gimpnet. Setup your JHBuild environment (1) ---------------------------------- [role="incremental"] - Bootstrap your JHBuild * `setenv MAKE gmake` (csh) or `export MAKE=gmake` (sh) * `./autogen.sh && gmake && gmake install` - Install patched libtool * Copy the `gnome-libtool` from the working tree of a GNOME ports, such as `devel/glib20`. * Add `os.environ['MAKE'\] = 'gmake LIBTOOL=/path/to/your/gnome-libtool'` to your `jhbuildrc` * Alternatively, you can run `jhbuild bootstrap libtool` and apply the following patch. Setup your JHBuild environment (2) ---------------------------------- [source,diff] --- libtool.m4.orig 2013-09-22 10:29:14.000000000 +0800 +++ libtool.m4 2013-09-22 11:48:33.000000000 +0800 @@ -2482,7 +2482,7 @@ version_type=freebsd-$objformat case $version_type in freebsd-elf*) - library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext} $libname${shared_ext}' + library_names_spec='${libname}${release}${shared_ext}.`expr $current - $age` ${libname}${release}${shared_ext} $libname${shared_ext}' need_version=no need_lib_prefix=no ;; Setup your JHBuild environment (3) ---------------------------------- - Remove all unneeded .la files in /usr/local/lib * However, you cannot remove the .la files of ImageMagick or it will be broken. * Instead of removing them, you can consider move them to other directories until this problem is fully solved by libtool or FreeBSD ports itself. - Follow the instructions on https://wiki.gnome.org/Projects/Jhbuild/FreeBSD * If you find that information on the wiki is wrong or outdated, please subscribe to the page and feel free to edit it. The end ------- * Thanks for your listening! * Any questions? // vim: set ft=asciidoc et ts=2 sts=2 sw=2: