New Installer Package for macOS

Dev
1

I have coded a makefile and updated a few of the source and build files of Graphviz to build a macOS installer package. This package installs graphviz dot and plugins into /usr/local/graphviz, adds /usr/local/graphviz/bin to the PATH using /etc/paths.d, and installs the Graphviz.app application in the /Applications folder.

I have the changes in a tarfile:
4232 Oct 4 18:17 .gitignore
1386 Oct 4 18:20 config-cmake.h.in
2140 Sep 3 17:18 lib/common/usershape.h
2201 Oct 5 15:29 macosx/prebuild/Makefile-packages.incl
5165 Aug 29 19:18 plugin/quartz/gvloadimage_quartz.c
2709 Sep 29 21:54 plugin/quartz/gvplugin_quartz.c
1932 Sep 29 21:54 plugin/quartz/gvplugin_quartz.h
17160 Aug 7 20:44 plugin/quartz/gvrender_quartz.c
1763 Oct 4 23:32 macosx/Distribution.xml
8395 Oct 4 22:49 macosx/package.mk

I also have an unsigned installer graphviz.pkg, which I can install on Apple Silicon Macs after running xattr -c graphviz.pkg.

I would like to submit these updates for the Graphviz team’s consideration.

2

ok! we normally review changes using GitLab’s “merge request” functionality.

3

I have a gitlab account, but gitlab sign in requires an extra step (2FA?) that I do not know how to get past:

Enter the code from your two-factor authenticator app. If you’ve lost your device, you can enter one of your recovery codes.

I am embarrassed to say that this process has foiled me, and I have not been successful attempting all the suggestions at resolution suggested by my searches on Duck Duck Go. Maybe there is a “start over” to create an account, but I don’t want to lose the “zosmac” user name. It is what I use for github.

I see that there is a graphviz mirror on github. Could I submit the pull/merge request there?
Thanks

4

It’s nice to get help with MacOS packaging after so long. What changes did you need to make to the drivers?

Maybe make a new account in gitlab for this purpose?

5

I consolidated the macos makefiles into one to create the macOS installer package. I updated the list of package dependencies, but only pull five down (autoconf, automake, libtool, pkg-config, and bison) to build dot and friends. I added heif output support to quartz. The makefile uses xcodebuild, pkgbuild, and productbuild to create the package installer.

I do not have an Apple Installer account, so I cannot sign the package for distribution. Make builds the package for local installation. I believe that productsign with an installer account will enable distribution. I am not sure how to distribute packages without having to clear the quarantine attribute (xattr -c) after download.

This actually all started when I discovered the dated Graphviz.app code in the graphviz repo. So I rewrote all the Objective-C and Interface Builder code to bring the code up to date. I can submit that code after this package build code.

6

I so far have two gitlab accounts that I can sign in with their id and password successfully, but cannot pass the “verification code” 2FA. I can even successfully change the password, but I have no idea what it is wanting for this verification code. All the set up for OAUTH apps and devices and recovery keys and whatever else does not guide me to a support link to resolve this, maybe because I only have free accounts on gitlab.

I know this isn’t your problem, so I am just going to reflect on the irony of how hard hackers have made life for the rest of us.

7

Thanks for working on this! Though I’m a little confused The Graphviz Autotools build system already supports building an installer package (see pkg target in the top level Makefile.am). Admittedly I’ve never run this and we don’t run it in CI, so it’s probably bitrotted, but it definitely exists. Are you saying you resurrected this? Or you wrote a completely new Makefile from scratch?

We’ve been trying for years to standardize on CMake as the sole Graphviz build system, but so far it has yet to reach partiy with the Autotools one. It too supports macOS though.

Given we have a very proactive Homebrew Graphviz maintainer, my preferred avenue for Graphviz distribution on macOS is through there. As far as I know, they support both x86 and ARM Macs. Did you find problems with this?

8

The existing macosx/build Makefiles use PackageMaker, which Apple no longer supports and which does not support code signing. The new makefile uses pkgbuild and productbuild. The productbuild command can sign, or one may sign the package with productsign.

I have an updated Makefile-packages.incl to reference all the latest versions of the dependencies.

I have a new makefile that installs only autoconf, automake, libtool, pkg-config, and bison prior to driving autogen, configure, and make to build dot and the plugins. It does not install optional features and packages. Quartz already provides several of these. I have not reviewed the overlap to determine what additional functionality from the optionals that Quartz lacks.

I did enable HEIF output in the Quartz plugin. I got as far as getting HEIF out of GD, using x265, libde265 and libheif, but the layouts it generated were skewed, as if some transform was applied. It also required graphs use the truecolor palette.

The makefile continues with xcodebuild to build the Graphviz.app and then creates the installer using pkgbuild and productbuild. I am not an Apple Developer Installer, so I cannot sign the packages for distribution and installation. Is that something that Homebrew can do? I am not conversant in Homebrew. Hopefully Homebrew can address the Apple quarantine applied to a distributed package.

I have the graphviz.pkg install dot and the plugins in /usr/local/graphviz and add /etc/paths.d/graphviz to point to /usr/local/graphviz/bin. Golang does something similar to isolate its install from all the other stuff going into /usr/local.

9

I have been able to use CMake to build Graphviz for the Mac if I specify -Dwith_gvedit=OFF. However, pkg-config in this case uses @rpath for some of the libraries, so that I have to specify DYLD_LIBRARY_PATH=/usr/local/graphviz/lib to run dot. I also ran into a problem with resolving zlib, for which I do have a fix in config-cmake.h.in.

Using CMake will streamline the build. I see that it also has CPack support for Xcode and so presumably be able to perform the package/installer build. I can look into that. Thanks.

10

Ah I was not aware of this. I don’t know much about the Apple ecosystem. I guess the updated process you describe should replace our existing use of PackageMaker then.

Ah so your makefile calls the existing Autotools build system. Got it.

I don’t know. I am not an Apple Developer Installer either. I guess Homebrew must have some way to make this work though, because apparently people use its Graphviz package successfully.

There’s almost certainly rpath problems. See

11

Thank you all for this feedback. I will revisit my changes. I have now made some more progress on using CMAKE as well. However, the post install configuration using calling dot -c does not create the config6 file. (When using the GNU tools dot -c does work). Not sure what dot -c looks for.

cmake -S. -B_build -DCMAKE_INSTALL_PREFIX=/usr/local/graphviz -P _build/cmd/dot/configure_plugins.cmake --trace-expand
Put cmake in trace mode, but with variables expanded.
/Users/zosmac/Developer/graphviz/_build/cmd/dot/configure_plugins.cmake(30):  set(ROOT /usr/local/graphviz )
/Users/zosmac/Developer/graphviz/_build/cmd/dot/configure_plugins.cmake(32):  if(APPLE )
/Users/zosmac/Developer/graphviz/_build/cmd/dot/configure_plugins.cmake(33):  set(ENV{DYLD_LIBRARY_PATH} /usr/local/graphviz/lib )
/Users/zosmac/Developer/graphviz/_build/cmd/dot/configure_plugins.cmake(38):  execute_process(COMMAND /usr/local/graphviz/bin/dot -c COMMAND_ERROR_IS_FATAL ANY )
12

I was able to work around the @rpath issue thusly:

cmake -S. -B_build -DCMAKE_MACOSX_RPATH=FALSE -DCMAKE_INSTALL_NAME_DIR=/usr/local/graphviz/lib -Dwith_gvedit=OFF --install-prefix=/usr/local/graphviz
13

I have defined new makefiles to build Graphviz for MacOS and package the output using pkgbuild and productbuild (successors to package maker).

I have divided the macosx/prebuild makefile into two:

  • macosx/prebuild/sonoma.mk builds the minimal set of GNU tools (autoconf, automake, libtool, and pkg-config, plus bison) to build Graphviz with Quartz on macOS.
  • macosx/prebuild/sonomax.mk that builds additional dependencies to include in Graphviz:
    • freetype, fontconfig, tiff, jpeg, png
    • pixman, cairo, pango
    • gts
    • x265, de265, heif, gd (will require updates to gd plugin to use heif)

gd layout refuses to configure.

The following additional optional dependencies are not built:

  • atk, gtk, flex, glib, gtk, iconv, webp, xft, xml2

These libraries are resolved from Xcode:

  • -lexpat -lffi -liconv -lpcre -lxml2 -lz

dot -v reports

dot - graphviz version 10.0.0~dev.20231015.0537 (20231015.0537)
libdir = "/usr/local/graphviz/lib/graphviz"
Activated plugin library: libgvplugin_dot_layout.6.dylib
Using layout: dot:dot_layout
Activated plugin library: libgvplugin_core.6.dylib
Using render: dot:core
Using device: dot:dot:core
The plugin configuration file:
	/usr/local/graphviz/lib/graphviz/config6
		was successfully loaded.
    render	:  cairo dot dot_json fig json json0 map mp pic pov ps quartz svg svg_inline tk xdot xdot_json
    layout	:  circo dot fdp neato nop nop1 nop2 osage patchwork sfdp twopi
    textlayout	:  textlayout
    device	:  bmp canon cgimage cmap cmapx cmapx_np dot dot_json eps exr fig gif gv heic icns ico imap imap_np ismap jp2 jpe jpeg jpg json json0 kitty kittyz mp pct pdf pic pict plain plain-ext png pov ps ps2 psd sgi svg svg_inline svgz tga tif tiff tk vt vt-24bit xdot xdot1.2 xdot1.4 xdot_json
    loadimage	:  (lib) bmp eps gif heic jpe jpeg jpg pdf png ps svg

For macOS, how much utility is there to build Graphviz with additional dependencies beyond what Quartz and dot provide?

14

I’m still unclear what the goal is here. Specifically what’s throwing me off:

  1. Why are you building the GNU tools from source? Graphviz doesn’t modify these in any way, so you should be able to use pre-built versions of all this. See commit a832ad72d437e13aad82a46581117cbfc822c4af in the Graphviz repo. The scripts this commit removed show a much simpler path to obtaining all these tools.
  2. Similarly, why are you building the Graphviz dependencies from source? We’ve never done that in CI.
  3. Why do you need a separate Makefile? I’m still not following why you cannot replace the pkg target in the existing top level Makefile.am.

You can also probably get away without “sonoma” in paths too. I don’t know much about macOS versions, but in recent times we’ve never tried to support anything other than the latest macOS.

I’ll have to leave this to others to comment.

15

Please grab a coffee or a beer, and feel free to dive in when ready. Answers to 1, 2, 3 follow, with further discussion below.

  1. The pkg target invokes a make file in macosx/build. This is what I am updating to use pkgbuild and productbuild rather than package maker.

2a) By CI to you mean Continuous Integration? I build from the cloned repo on my MacBook and found that I need to start with autogen.sh, which employs the GNU tools. These are not part of macOS or the Command Line Tools. I do not use Homebrew or MacPorts. The make file scripts in macosx/prebuild provided the template for downloading and installing the GNU Tools on the Mac.

2b) I am not sure what all the dependencies built in the old macosx/prebuild/Makefile.* are for, (fontconfig, pango, cairo, gdlib,…), and if Quartz is sufficient for macOS deployments of Graphviz, then the Makefile can stop with the GNU tools (and Bison).

  1. I could update the pkg target directly in Makefile.am by pulling over what I put in the macosx/build make file. The macosx/build make file has targets to
  • write a PATH directive for /usr/local/graphviz/bin to /etc/paths.d/graphviz
  • define a postinstall script to run dot -c
  • pkgbuild Graphviz to deploy to /usr/local/graphviz
  • xcodebuild the objective-c Graphiz app
  • pkgbuild Graphviz.app to deploy to /Applications
  • productbuild to assemble this all into a single installer package

Further discussion:

Anyway, I perhaps have ventured a bit into the weeds here. May I offer how I got here, and hopefully you can get me back on track.

For several years I have used Graphviz dot to render SVG of process diagrams, after discovering how Go used dot to diagram CPU and memory usage. I built Graphviz from source on my Mac using the instructions at https://graphviz.org/download/source/. The only requirement for this build is to install the macOS Command Line Tools.

My first possible detour is related to the Graphviz.app. Is this app still relevant? If so, please refer to Note 1 below. Otherwise, perhaps all this code can be removed from the repo.

The old make files in macosx/prebuild install the GNU tools, so I set about writing a “gnutools.mk” to download and install the GNU tools to streamline my workflow for building Graphviz from the repo.

The make files in macosx/prebuild led me on my second possible detour. These make files install a number of other Graphviz dependencies with which I am unfamiliar. If these dependencies are useful for the macOS version of Graphviz, see Note 2.

In the macosx/build make files the obsolete package maker tool is used to create an installer for macOS. This is where I focused on the update to use pkgbuild and productbuild. This is the make file that Makefile.am references to build the installer package with the make -C flag. If updating the pkg target to invoke pkgbuild and productbuild directly is the goal, please see Note 3. Finally, see Note 4 regarding signing of installer packages.

As an aside, I have also been exploring how to build Graphviz with CMake. (you helped me with the LIBZ/ZLIB naming issue). I believe that this would preclude the need for the GNU tools. CMake also supports CPack which may also facilitate the packaging.

Thank you.

Note 1:

Last spring I happened to notice the macosx folder and the old Objective-C code for a Graphviz.app. I rewrote this code using Xcode to update it for newer capabilities of Objective-C such as ARC, and to resolve compiler and deprecation warnings and errors. It became a substantial rewrite, including a few changes to the Quartz plugin to add HEIC support.

As I was making these code changes, I started to wonder whether this would be of utility to anyone else. So I cloned the Gitlab repo and started building from that. In my experience, to build from the repo I need to start with the autogen.sh script. Starting the build with this script calls for installing the GNU tools, as well as Bison, as the Bison installed with the Command Line Tools is too old.

So at that point, I was installing the GNU tools, building Graphviz --with-quartz, then opening the macosx/graphviz.xcodeproj in Xcode, and building the app in Xcode.

Note 2:

I updated macosx/prebuild/Makefile-packages-incl to reference the latest versions of the GNU tools and other dependencies and created two prebuild make files for macOS: gnutools.mk for the GNU tools and dependencies.mk for the additional dependencies. I named these with a .mk extension so that VSCode recognizes them for syntax highlighting.

Note 3:

The new macosx/build/Makefile uses xcodebuild, pkgbuild, and productbuild to create an installer package. It also builds Graphviz. In this way, the package contains everything for a deployment. It drops all of Graphviz into /usr/local/graphviz (to isolate it from /usr/local), creates an /etc/paths.d/graphviz file to add /usr/local/graphviz/bin to the PATH, and installs the Graphviz.app into /Applications. In this way, a user can download the package and run the installer, Theoretically then, Graphviz could even be distributed through the Mac App store.

I build Graphviz from the macosx/build/Makefile with a staging location for --prefix and DESTDIR for the embedded install targets. This may seem a bit involved, but in this way the installer package can all be assembled from a staging location rather than from /usr/local. The installer itself then deploys into /usr/local/graphviz

I deploy to /usr/local/graphviz to make it easier for me to rerun the builds without having to selectively remove all the Graphviz deployed files. By using a staging location, one can build the package without inadvertently populating it with other files the developer may have in /usr/local.

Note 4:

To distribute a macOS installer package requires that it be signed with an Apple Developer installer enabled account. I am not an Apple Developer and I do not work for a company that could provide a vetted Apple Developer signature.

| smattr
October 23 |

  • | - |

I’m still unclear what the goal is here. Specifically what’s throwing me off:

  1. Why are you building the GNU tools from source? Graphviz doesn’t modify these in any way, so you should be able to use pre-built versions of all this. See commit a832ad72d437e13aad82a46581117cbfc822c4af in the Graphviz repo. The scripts this commit removed show a much simpler path to obtaining all these tools.
  2. Similarly, why are you building the Graphviz dependencies from source? We’ve never done that in CI.
  3. Why do you need a separate Makefile? I’m still not following why you cannot replace the pkg target in the existing top level Makefile.am.

You can also probably get away without “sonoma” in paths too. I don’t know much about macOS versions, but in recent times we’ve never tried to support anything other than the latest macOS.

zosmac:

how much utility is there to build Graphviz with additional dependencies beyond what Quartz and dot provide?

I’ll have to leave this to others to comment.

16

Yes.

OK, but why? Any workflow that builds all this stuff from source is a nonstarter for merging into the main repo.

We should probably remove these. They have not been used in years (some in over a decade), and this thread demonstrates they are leading people astray.

17

For whom is a macOS package installer targeted?

Is the macOS Cocoa app (Graphviz.app) expected to be part of the install on macOS?

| smattr
October 24 |

  • | - |

zosmac:

By CI to you mean Continuous Integration?

Yes.

zosmac:

I do not use Homebrew or MacPorts.

OK, but why? Any workflow that builds all this stuff from source is a nonstarter for merging into the main repo.

zosmac:

The make file scripts in macosx/prebuild provided the template for downloading and installing the GNU Tools on the Mac.

We should probably remove these. They have not been used in years (some in over a decade), and this thread demonstrates they are leading people astray.

18

This is correct. I’m really sorry, we have a lot of vestigial code from the old MacOS build when there was no Homebrew or MacPorts and we just copied linux source tarballs and built them with patches.

I would strongly favor building and iinstalling Graphviz.app because it’s very Maclike and the audience for a standalone installer would probably appreciate this. Otherwise people can go through Macports or Homebrew to get Graphviz anyway.

19

The make pkg build depends on make dist.

make dist creates a tar file. On my Mac, running make dist fails for gvedit, which depends on Qt.

What is gvedit? I am not familiar with Qt. Is that something one would expect to use on a Mac?

Thanks

20

Gvedit is a GUI for editing Graphviz graphs.

Yes, something is busted here. I just tried uninstalling the Qt deps in my Linux environment and make dist fails for me too. make dist without Qt fails (#2463) · Issues · graphviz / graphviz · GitLab