The most interesting projects are:

• pandoc
• git-annex
• gitit
• yi
• xmonad

Thanks!

If you happen to be involved in using/developing haskell-powered software you might like to answer our poll on that matter.

Thanks in advance!

Here is a small incomplete HOWTO for gentoo users on how to build a crosscompiler running on x86_64 host targeted on ia64 platform.

It is just an example. You can pick any target.

First of all you need to enable haskell overlay and install host compiler:

# GHC_IS_UNREG=yeah emerge -av =ghc-7.6.1

The GHC_IS_UNREG=yeah bit is critical. If we won’t do it GHC build system will try to build registerised stage1 (which is a crosscompiler already).

Not setting GHC_IS_UNREG will break for a set of problems:

• GHC will try to optimize generated bitcode with llvm‘s optimizer which will produce x86_64 instructions, not ia64.

• GHC will try to run (broken on ia64) object splitter perl script: ghc-split.lprl.

The rest is rather simple:

# crossdev ia64-unknown-linux-gnu
# ia64-unknown-linux-gnu-emerge sys-libs/ncurses virtual/libffi dev-libs/gmp
# ln -s ${haskell_overlay}/haskell/dev-lang/ghc ${cross_overlay}/ia64-unknown-linux-gnu/ghc
# cd ${cross_overlay}/ia64-unknown-linux-gnu/ghc
# EXTRA_ECONF=--enable-unregisterised USE=ghcmakebinary ebuild ghc-9999.ebuild compile

It will fail as the following command tries to run ia64 binary on x86_64 host:

libraries/integer-gmp/cbits/mkGmpDerivedConstants > libraries/integer-gmp/cbits/GmpDerivedConstants.h

I’ve logged-in to ia64 box and ran mkGmpDerivedConstants to get a GmpDerivedConstants.h. Added the result to the ${WORKDIR} and reran last command.

After the build has finished I’ve got corsscompiler:

sf ghc-9999 # "inplace/bin/ghc-stage1" --info
 [("Project name","The Glorious Glasgow Haskell Compilation System")
 ,("GCC extra via C opts"," -fwrapv")
 ,("C compiler command","/usr/bin/ia64-unknown-linux-gnu-gcc")
 ,("C compiler flags"," -fno-stack-protector  -Wl,--hash-size=31 -Wl,--reduce-memory-overheads")
 ,("ld command","/usr/bin/ia64-unknown-linux-gnu-ld")
 ,("ld flags","     --hash-size=31     --reduce-memory-overheads")
 ,("ld supports compact unwind","YES")
 ,("ld supports build-id","YES")
 ,("ld is GNU ld","YES")
 ,("ar command","/usr/bin/ar")
 ,("ar flags","q")
 ,("ar supports at file","YES")
 ,("touch command","touch")
 ,("dllwrap command","/bin/false")
 ,("windres command","/bin/false")
 ,("perl command","/usr/bin/perl")
 ,("target os","OSLinux")
 ,("target arch","ArchUnknown")
 ,("target word size","8")
 ,("target has GNU nonexec stack","True")
 ,("target has .ident directive","True")
 ,("target has subsections via symbols","False")
 ,("Unregisterised","YES")
 ,("LLVM llc command","llc")
 ,("LLVM opt command","opt")
 ,("Project version","7.7.20130118")
 ,("Booter version","7.6.1")
 ,("Stage","1")
 ,("Build platform","x86_64-unknown-linux")
 ,("Host platform","x86_64-unknown-linux")
 ,("Target platform","ia64-unknown-linux")
 ,("Have interpreter","NO")
 ,("Object splitting supported","NO")
 ,("Have native code generator","NO")
 ,("Support SMP","NO")
 ,("Tables next to code","NO")
 ,("RTS ways","l debug  thr thr_debug thr_l thr_p ")
 ,("Dynamic by default","NO")
 ,("Leading underscore","NO")
 ,("Debug on","False")
 ,("LibDir","/var/tmp/portage/cross-ia64-unknown-linux-gnu/ghc-9999/work/ghc-9999/inplace/lib")
 ,("Global Package DB","/var/tmp/portage/cross-ia64-unknown-linux-gnu/ghc-9999/work/ghc-9999/inplace/lib/package.conf.d")
 ]

# cat a.hs
main = print 1
# "inplace/bin/ghc-stage1" a.hs -fforce-recomp -o a
[1 of 1] Compiling Main             ( a.hs, a.o )
Linking a ...
# file a
a: ELF 64-bit LSB executable, IA-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.16, not stripped
# LANG=C ls -lh a
-rwxr-xr-x 1 root portage 24M Jan 20 02:24 a
on ia64:
$ ./a
1

Results:

• It’s not that hard to build a ghc with some exotic target if you have gcc there.

• mkGmpDerivedConstants needs to be more cross-compiler friendly It should be really simple to implement, it only queries for data sizes/offsets. I think autotools is already able to do it.

• GHC should be able to run llvm with correct -mtriple in crosscompiler case. That way we would get registerised crosscompiler.

Some TODOs:

In order to coexist with native compiler ghc should stop mangling —-target=ia64-unknown-linux-gnu option passed by user and name resulting compiler a ia64-unknown-linux-gnu-ghc and not ia64-unknown-linux-ghc.

That way I could have many flavours of compiler for one target. For example I would like to have x86_64-pc-linux-gnu-ghc as a registerised compiler and x86_64-unknown-linux-gnu-ghc as an unreg one.

And yes, they will all be tracked by gentoo’s package manager.

Some notes and events in the overlay:

• ghc-7.6.1 is available for all major arches we try to support
• a few ebuilds of overlay were converted to EAPI=5 to use subslot depends (see below)
• we’ve got working ghc-9999 ebuild with shared libraries by default! (see below)

ghc-7.6

That beast brought two major problems to it’s users:

1. Prelude.catch gone away and is called ‘System.IO.Error.catchIOError’ now
2. directory package broke interface to existing function ‘getModificationTime’ without old compatible variant.

While the first breakage is easy to fix by something like:

#if MIN_VERSION_base(4,6,0)
catch :: IO a -> (IOError -> IO a) -> IO a
catch = System.IO.Error.catchIOError

(or just switch to extensible-exceptions package if you need support for really old ghc versions).

The second one is literally a disaster

-getModificationTime :: FilePath -> IO ClockTime
+getModificationTime :: FilePath -> IO UTCTime

It is not as straightforward and "fixes" in various packages break PVP in a very funny way.

Look at this example.

Now that package has random signature type depending on which directory version it decided to build against.

TODO: find a nice and simple ‘:: ClockTime -> IO UTCTime’ compatibility function to end that keep creeping mess. (I wish the directory package to provide that).

Okay. Enough ranting.

EAPI=5

Some of experienced gentoo haskell users already know about the magic haskell-updater tool written by Ivan to fix the mess after ghc upgrade or some base library upgrade.

Typical symptom of broken libraries is the similar ghc-pkg check result:

There are problems in package data-accessor-monads-fd-0.2.0.3:
  dependency "monads-fd-0.1.0.4-830f79a91000e99707aac145b972f786" doesn't exist
There are problems in package LibZip-0.10.2:
  dependency "mtl-2.0.1.0-b1b6de8085e5ea10cc0eb01054b69110" doesn't exist
There are problems in package jail-0.0.1.1:
  dependency "monads-fd-0.1.0.4-830f79a91000e99707aac145b972f786" doesn't exist

Why it happens?

Well, ghc’s library ABI depends on ABIs on all the libraries it uses. It has quite nasty consequences.

Once you upgrade a library you need to:

1. rebuld all the reverse dependencies
2. and their reverse dependencies (recursive)

The first point can be solved by EAPI 5 so called SUBSLOT feature.

The second one is not solved yet, but i was said is planned for EAPI=6. Thus you will still need to use haskell-updater time to time.

Anyway, I’ve bumped binary package today and to show how portage picks all it’s immediate users:

# emerge -av1 dev-haskell/binary

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  r  U ~] dev-haskell/binary-0.6.4.0:0/0.6.4.0::gentoo-haskell [0.6.2.0:0/0.6.2.0::gentoo-haskell] USE="doc hscolour {test} -hoogle -profile" 0 kB
[ebuild  r  U ~] dev-haskell/sha-1.6.1:0/1.6.1::gentoo-haskell [1.6.0:0/1.6.0::gentoo-haskell] USE="doc hscolour -hoogle -profile" 2,651 kB
[ebuild  r  U ~] dev-haskell/zip-archive-0.1.2.1-r2:0/0.1.2.1::gentoo-haskell [0.1.2.1-r1:0/0.1.2.1::gentoo-haskell] USE="doc hscolour {test} -hoogle -profile" 0 kB
[ebuild  rR   ~] dev-haskell/data-binary-ieee754-0.4.3:0/0.4.3::gentoo-haskell  USE="doc hscolour -hoogle -profile" 0 kB
[ebuild  rR   ~] dev-haskell/dyre-0.8.11:0/0.8.11::gentoo-haskell  USE="doc hscolour -hoogle -profile" 0 kB
[ebuild  rR   ~] dev-haskell/hxt-9.3.1.1:0/9.3.1.1::gentoo-haskell  USE="doc hscolour -hoogle -profile" 0 kB
[ebuild  rR   ~] dev-haskell/hashed-storage-0.5.10:0/0.5.10::gentoo-haskell  USE="doc hscolour {test} -hoogle -profile" 0 kB
[ebuild  rR   ~] dev-haskell/dbus-core-0.9.3-r1:0/0.9.3::gentoo-haskell  USE="doc hscolour -hoogle -profile" 0 kB
[ebuild  rR   ~] dev-haskell/hoogle-4.2.14:0/4.2.14::gentoo-haskell  USE="doc fetchdb hscolour -fetchdb-ghc -hoogle -localdb -profile" 0 kB
[ebuild  rR   ~] www-apps/gitit-0.10.0.2-r1:0/0.10.0.2::gentoo-haskell  USE="doc hscolour plugins -hoogle -profile" 0 kB
[ebuild  r  U ~] dev-haskell/yesod-auth-1.1.1.7:0/1.1.1.7::gentoo-haskell [1.1.1.6:0/1.1.1.6::gentoo-haskell] USE="doc hscolour -hoogle -profile" 17 kB
[ebuild  rR   ~] dev-haskell/yesod-1.1.4:0/1.1.4::gentoo-haskell  USE="doc hscolour -hoogle -profile" 0 kB

Total: 12 packages (4 upgrades, 8 reinstalls), Size of downloads: 2,668 kB

Would you like to merge these packages? [Yes/No]

I would like to rebuild all the sha (and so on) revdeps as well, but EAPI can’t express that kind of depends yet.

The EAPI=5 ebuild slowly drift to main portage tree as well.

ghc-9999

The most iteresting thing!

With great Mark’s help we now have live ghc ebuild right out of gti tree!

One of the most notable things is the dynamic linking by default.

# ldd `which happy` # ghc-7.7.20121116
    linux-vdso.so.1 (0x00007fffb0bff000)
    libHScontainers-0.5.0.0-ghc7.7.20121116.so => /usr/lib64/ghc-7.7.20121116/containers-0.5.0.0/libHScontainers-0.5.0.0-ghc7.7.20121116.so (0x00007fe616972000)
    libHSarray-0.4.0.1-ghc7.7.20121116.so => /usr/lib64/ghc-7.7.20121116/array-0.4.0.1/libHSarray-0.4.0.1-ghc7.7.20121116.so (0x00007fe6166d0000)
    libHSbase-4.6.0.0-ghc7.7.20121116.so => /usr/lib64/ghc-7.7.20121116/base-4.6.0.0/libHSbase-4.6.0.0-ghc7.7.20121116.so (0x00007fe615df9000)
    libHSinteger-gmp-0.5.0.0-ghc7.7.20121116.so => /usr/lib64/ghc-7.7.20121116/integer-gmp-0.5.0.0/libHSinteger-gmp-0.5.0.0-ghc7.7.20121116.so (0x00007fe615be6000)
    libHSghc-prim-0.3.0.0-ghc7.7.20121116.so => /usr/lib64/ghc-7.7.20121116/ghc-prim-0.3.0.0/libHSghc-prim-0.3.0.0-ghc7.7.20121116.so (0x00007fe615976000)
    libHSrts-ghc7.7.20121116.so => /usr/lib64/ghc-7.7.20121116/rts-1.0/libHSrts-ghc7.7.20121116.so (0x00007fe615715000)
    libc.so.6 => /lib64/libc.so.6 (0x00007fe61536c000)
    libHSdeepseq-1.3.0.1-ghc7.7.20121116.so => /usr/lib64/ghc-7.7.20121116/containers-0.5.0.0/../deepseq-1.3.0.1/libHSdeepseq-1.3.0.1-ghc7.7.20121116.so (0x00007fe615162000)
    libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007fe614ef4000)
    libffi.so.6 => /usr/lib64/libffi.so.6 (0x00007fe614cec000)
    libm.so.6 => /lib64/libm.so.6 (0x00007fe6149f2000)
    librt.so.1 => /lib64/librt.so.1 (0x00007fe6147ea000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007fe6145e6000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fe616d41000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe6143ca000)

$ ls -lh `which pandoc` # ghc-7.7.20121116
-rwxr-xr-x 1 root root 6.3M Nov 16 16:38 /usr/bin/pandoc
$ ls -lh `which pandoc` # ghc-7.4.2
-rwxr-xr-x 1 root root 27M Nov 18 17:46 /usr/bin/pandoc

Actually, the whole ghc-9999 installation is 150MB smaller, than ghc-7.4.1 on amd64.

Quite a win!

And as a side effect revdep-rebuild (or portage’s FEATURES=preserved-rebuild) can note (and fix) introduced breakages due to upgrades!

Work on the ghc cross-compilation in the ebuild slowly continues (needs some upstream fixes to support toolchains inferred from build/host/target triplets).

Have fun!

• ghc-7.2 and 7.4 were released by GHC HQ
• hackage got ever increasing ton of package bumps
• overlay got some of those updates
• cabal test suite by Thomas Tuegel became more popular
• and many others!

ghc-7.4

Yes! ghc-7.4.1 is now in main tree (but masked). We’ve just finished building ghc binaries.

Thus far ghc on gentoo is expected to work on:

alpha, amd64, ppc, ppc64, sparc and x86.

It should also work on arm, but you will have to bootstrap it yourself from ghc-7.0.4 binary. It took me 2 days of CPU time. I’ll try to build proper armv5 binary a bit later.

ia64 support was not restored yet: it needs either patching gcc to support -mcmodel-medium or hacking ghc C Codegen to generate less global objects.

While things generally work in overlay some ~arch things in main tree are still broken against ghc-7.4.1. Please, drop a note on #gentoo-haskell when you find such one. Thanks in advance!

Overlay contributors

For the past year we have got new contributions:

gentoo-haskell $ git shortlog -s -n --since='last year'
2048  Mark Wright
 786  Sergei Trofimovich
  43  Alexander Vershilov
  25  Lennart Kolmodin
   7  Dmitry Dzhus
   5  Mike Limansky
   4  Emil Karlson
   4  jay groven
   3  Florian Schmaus
   3  Tom Prince
   3  exbb2
   2  Jacques Kvam
   2  Oleg 'silver' Osipovich
   1  James Cloos
   1  Jared Hance
   1  Keshav Kini
   1  Naohiro Aota
   1  Oleg Osipovich
   1  Perttu Luukko
   1  Ulya Fokanova
   1  chiro

First of all, a lot of new names!

Mark became the gentoo developer and now helps haskell and science teams to keep the things rolling.

Alexander not only helps us to keep things up-to-date but also managed to tweak hackport to generate USE=profile style depends like the following:

RDEPEND="=dev-haskell/conduit-0.2*[profile?]
         =dev-haskell/stm-2.2*[profile?]
         =dev-haskell/stm-chans-1.3*[profile?]
         =dev-haskell/transformers-0.2*[profile?]
         >=dev-lang/ghc-6.10.1"
DEPEND="${RDEPEND}
        >=dev-haskell/cabal-1.8"

It will help users get less build-time failures when they enable profiling only for a subset of haskell libraries (gentoo bug #341795).

I’ve added a feature to hackport which helps us spy on hackage updates: hackport status --from-hackage.

The following (clickable) sample output should be selfdescriptive:

You can see there leksah, unordered-containers, git-annex and other updates.

Some upstreams don’t upload their releases on hackage. Think about it: by uploading you get more users for free, ease distro packaging and get more feedback! Thanks!

We’ve got various repositories, some larger with several contributors and some smaller with only one developer. All repos were based on the darcs revision control system.

You may browse the new repos at our new home.

mkdir overlay.git && cd overlay.git
darcs-to-git ../overlay
git commit --allow-empty -m "phony" # hack, described later
darcs-to-git ../overlay

Running: ["git", "log", "-n1", "--no-color"]
fatal: bad default revision 'HEAD'

mkdir overlay.git && cd overlay.git
(cd ../overlay ; darcs-fastconvert export) | git fast-import

git filter-branch --tree-filter \
      'rm -rf ignore-this-file and-this-directory' HEAD
git filter-branch --prune-empty -f

mkdir keyword-stat.git && cd keyword-stat.git
git init
( cd ../keyword-stat ; darcs-fastconvert export ) \
    | git fast-import

It will require some additional actions for overlay users:

layman -f
layman -d haskell
layman -a haskell

Our move was stimulated by a couple of events:

• code.haskell.org was not very reliable, and

• as you might notice haskell overlay was (and still is) inaccessible since last week. Current status of code.haskell.org can be tracked here.

We’ve got ghc-6.12.3 stable on x86, amd64 and sparc arches! (more to come)

What does it mean for end user?

You’ll get latest haskell compiler and will be able to taste it’s new features! We also got rid of hacky ghc-updater and switched to haskell-updater.

Unfortunately, some haskell packages in main tree might stop building on new compiler. Please report them as problematic to gentoo bugzilla or drop a line in #gentoo-haskell on freenode IRC network.

I’ve decided to look back and estimate timeframe it took us to deliver ghc for you:

• 12 Oct 2009 – GHC-6.12 RC1 appears in public
• 2 Nov 2009 – GHC-6.12 RC1 hits gentoo-haskell overlay
• 12 Jun 2010 – GHC-6.12.3 is released
• 26 Sep 2010 – GHC-6.12.3 goes stable on amd64

So, it took us almost 4 months.

The major problems were:

• Resurrect GHC on exotic architectures (we have 10 patches on top of vanilla ghc!). Unfortunately, hppa support was lost.
• Fix packages breaking with 6.10/6.12 branch of ghc (tons of them)

Brave souls can try to install ghc-7-rc2 (aka 7.0.0.20101028) from overlay (currently masked). It has no base-3 (deprecated in ghc-6.10), so you’ll have great chance to become a contributor to various haskell projects!

Enjoy!

ghc-updater is a bash script that was based long ago on python-updater. However, whilst python-updater was updated to let users use alternative package managers like Paludis or PkgCore, it wasn’t as simple to upgrade ghc-updater as we bundle it with dev-lang/ghc. So, we (meaning I :p ) decided to split off a new haskell-updater package (with a deliberate name change to avoid problems with name clashes). At first this was just going to be based off the newer versions of python-updater, but this approach was soon abandoned (I had trouble finding my way through what it was doing).

Thus, haskell-updater is unique as far as I know amongst the various app-admin/*-updater packages (well, the only two that are there are python-updater and emacs-updater :p ) in that it is written in the languages whose packages it aims to update and rebuild. Using Haskell for haskell-updater gives us several advantages over the “traditional” bash-based kludge:

• It uses our language of choice; this means we’re more likely to be interested in it and maintain it (and be able to maintain it!) in future.
• A more modular design makes it easier to split apart parts of the code rather than a one-file-fits-all bash script.
• Ability to use the Cabal library (more on why this is a good thing later).
• Speeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeed! After all, all Gentoo-ers are ricers, aren’t we? :p Seriously, haskell-updater takes roughly 2s for me to run (whilst doing more! see below), ghc-updater takes 27s and I killed python-updater after 8.5 minutes :s
• Have a piece of software written in Haskell that Don Stewart isn’t going to brag about having it available in Arch

Now, haskell-updater doesn’t just find packages installed with previous versions of GHC like ghc-updater did; it also finds broken packages, making it equivalent to revdep-rebuild/reconcilio/etc. for Haskell packages (though just for libraries, since at the moment GHC creates statically-linked binaries). This has become a bigger problem in the last few years as the number of Haskell libraries has almost exploded (especially after the base library being split up). Until now, however, users have had to manually run “ghc-pkg check” and build the corresponding packages by hand (otherwise you face the dreaded Diamond Dependency Problem). However, version 1.6 of the Cabal library includes support for parsing the output of “ghc-pkg check”, so we’re able to use this to have haskell-updater find these broken packages for you as well!

haskell-updater has several other incremental advantages over ghc-updater (supports slotted packages properly; able to find packages installed with previous versions of GHC even if someone manually went and deleted the old GHC directory when they shouldn’t have, etc.). As such, we highly recommend that people install it and try it out. Note, however, that it requires one of the 6.10 series of GHC releases or higher to work (technically it doesn’t as long as you install the necessary libraries yourself; however, we’ve stated this in the ebuild to try and avoid dependency problems when upgrading GHC). With version 6.10.4, it has completely replaced ghc-updater (ghc-updater is still shipped with previous versions) as the used updating tool.

Extra features we’re considering adding in future releases:

• Allow user-defined package managers (in case of custom scripts, etc.).
• The ability to print out the command to rebuild the packages rather than actually running it.
• Detect and fix packages that didn’t get re-registered by ghc-pkg when re-building the same version of GHC (this seems to be a problem when using Paludis).
• Adding colours to the output

Note that haskell-updater is not available on HackageDB like most other Haskell software; this is to avoid name-pollution there, and because we don’t think non-Gentoo users are going to be interested in it (and Gentoo users will probably have it automatically installed for them anyway).

#gentoo-haskell on freenode