I’ve written a C file that automatically generate a .dot file in order to visualize some quadtrees.
With little size, everything is perfect. So, I’ve tried woth more huge size with the command line “dot -Tpdf visualise.dot -o quadtree.pdf”. There is 424 321 nodes which is quite huge, I know…
In return, I’ve got a segfault… If someone could help me, it would be very appreciate.
Here is the .dot file :
visualise.dot (2.8 MB)
What OS? What version of graphviz?
While you’re free to run graphviz over large graphs like this, I have to wonder whether a pdf with 424k nodes in it will be useful? You’ll have to scroll a lot… I wonder if you might be better, as quadtrees natively represent x,y space, might be better plotted onto a square?
I’m using a mac, with graphviz version 12.1.0.
Yes, I’ll have to scroll a bit, but that is not so embarrasing, a priori, for it is quite rapid
Just general advice: graphs (or other visualizations) with many tens of thousands of objects or larger usually requires some careful engineering throughout the pipeline of layout, rendering, and interaction. For example the SVG document model in most web browsers runs out of steam at that level, so other rendering tricks may be needed. (In an unrelated consulting project, we were able to implement heat maps of 500K to 1M cells, including custom text labels for cells, using D3.js, HTML Canvas, and offscreen asynchronous rendering of the text.) N^2 network layout may bog down and use a lot of memory.
We didn’t focus a lot of effort on this in Graphviz, except sfdp is aimed at that level, and the Topological Fisheye Viewer sometimes gets more credit than it probably deserves. See this obscure demo video.
It may be fruitful to look for large tree viewers, like large phylogenic tree viewers from biology, or viewers from bioinfomatics.
If you don’t need to see explicit edges, then layout and viewing with tSNE or UMAP may be effective.
Hmm, gc says there are 21217 nodes & 21216 edges
If you remove minlen=10 from the edge attributes, dot will finish (correctly) in just a few seconds - if you produce svg.
It took ~2 minutes to produce pdf. Then Acrobat said that the result was too wide to fully display.
minlen is a measure of ranks, not points or inches.
Nice graph. I’d post it, but it is quite large.
We’ve had this debate numerous times and I’m still puzzled by this claim. I (not regularly, but occasionally) open SVGs with >100000 objects in Chrome on Linux. While a graph this size is often quite hard to read, Chrome has no trouble scrolling around it and zooming in and out. I guess discrepancies in our OS+browser combination is causing this large difference in experience?
Opened a bug for this, segfault when PDFing large graph (#2646) · Issues · graphviz / graphviz · GitLab
You may be right. It’s been 5+ years since I’ve looked into this. (I’m mostly on Safari but I built a scalable pixel heatmap viewer in Chrome / d3 / canvas on MacOS). I’m glad to hear this. It would be interesting to confirm if Chrome handles that level of scale with graph layouts, with default graphviz rendering like text labels, ellipse node shapes, Beziers for drawing edges etc. and I’ll stop spreading misinformation. (His 400K node tree would seem to result in at least 1.6M SVG objects: a polygon or ellipse per node with an associated text label, and a line and an arrowhead per edge?)
Thank you for mentioning this.
TBH my macOS knowledge is out-of-date too. The last time I tried something like this on macOS (~2019), the performance of Safari on this workload was very poor. I would not be surprised if this has improved significantly in all browsers in recent years thanks to people’s desire to produce interactive debugging aids (e.g. Flame Graphs) as SVGs, that can get pretty large.