Format of changelog

Today, the changelog is a plain text file without the possibility link to issues or other stuff. It is also not possible to link to sections within it from other sources. I would like to change this to a changelog in markdown format and rename it to CHANGELOG.md according to https://keepachangelog.com/en/0.3.0/. I use this for my projects: https://github.com/magjac/d3-graphviz/blob/master/CHANGELOG.md and https://github.com/magjac/graphviz-visual-editor/blob/master/CHANGELOG.md and find it very useful.

What do you think?

OK but be careful of the rpm packager which has certain expectation of the format. Date string in particular

Oh, I see. Maybe this is not such a good idea then? How can I test it against the rpm package manager?

Sounds good to me. Changelists are full of bullets and links, both of which are easier to write in markdown than plaintext.

1 Like

I misspoke. I was thinking of the format of %changelog in the rpm spec files ( generated from redhat/graphviz.spec.fedora.in )

However, it would-be-nice if we could generate both a ChangeLog file and the %changelog in .spec from a common source.

The changelog of an installed rpm can be seen with (e.g;):

$ rpm -q --changes graphviz

  • see git log
    (This automatically-replaced single-entry is used for frequent
    updates in the unstable development series of releases.
    A properly constructed changelog entry is a requirement
    for stable releases.)

Done: https://gitlab.com/graphviz/graphviz/-/merge_requests/1394

Here’s a direct link if you want to have a look at it: https://gitlab.com/magjac/graphviz/-/blob/convert-changelog-to-markdown/CHANGELOG.md

At the bottom of the source file there are some entries that are not visible in the rendered page, but are essential for the version links to the GitLab commit comparisons to work:

[Unreleased]: https://gitlab.com/graphviz/graphviz/compare/2.44.0...master
[2.44.0]: https://gitlab.com/graphviz/graphviz/compare/2.42.4...2.44.0
[2.42.4]: https://gitlab.com/graphviz/graphviz/compare/2.42.3...2.42.4
[2.42.3]: https://gitlab.com/graphviz/graphviz/compare/2.42.2...2.42.3

A new entry must be added here when a release is made. It’s very easy to forget.

From https://gitlab.com/graphviz/graphviz/-/merge_requests/1393#note_354058121:

Yes, I meant to write something about it here, but it was way past bedtime. Your suggestion LGTM. Perhaps with the clarification that the user in this case also includes a user who wants to build the software, but not us maintainers per se. E.g. stuff that relates only to the CI pipeline does not end up there. My reasoning is that the changelog must be short enough to be useful to external users. More hard-core users, such as maintainers and even contributors are expected to use be able to use commit messages, issues and merge requests.

As an example, when I rewrote the latest release, I removed references to my Docker image merge requests since Docker images are not (yet anyway) part of the “product”. I also removed references to modifications of the CI/CD pipeline and redundant merge requests, i.e. those that fixed an issue for which the issue itself was mentioned.

OK, so to choose some examples:

Does this sound like the correct intuition? I deliberately picked two changes that are pretty close together in terms of what type of “user” is likely to see the impact.

To channel some of @ellson’s comments, I think we have three categories of “user”:

  1. end user downloading e.g. an rpm
  2. end user building from source
  3. developer hacking in the Graphviz tree

Changes deserve a changelog entry if they impact (1) or (2) or both. Changes can be omitted from the changelog if they only impact (3), who can get their knowledge from the commit log. Do I have that right?

Yes, that’s an accurate interpretation of what I meant, but I’m definitely open to discuss it.

You forgot two user categories:

0a. end user using Graphviz without knowing how they got it.
0b. end user using Graphviz without knowing it (probably as part of another application)

Somewhat meant as a joke, but when talking about users, it’s important to talk also about what they are using: the DOT language, the CLI tools, the binaries, the installers, the build system, the source code, the commits, the merge requests, the issues, the test results, the GitLab meta-data etc.

Since we intend to maintain only one changelog, we must draw the line somewhere.

Ah yes. Browsing the open issues has been quite a wild ride of “is this person even talking about Graphviz…?”

1 Like

To elaborate further on what (I think) should and should not go into the changelog:

  • #1737 should go into the changelog, since it’s a problem that was present in the latest release.
  • #1735 should not go into the changelog, since it’s a problem that wasn’t present in the latest release. No user of the released versions (binaries or portable source) have had a chance to be hit by it and users building the software from HEAD are expected to keep track of issues and commits themselves.

Objections?

I agree. For the same reasons, I had assumed #1734 would not get a changelog entry.

1 Like

I like the part about people using Graphviz without knowing where they got it, or that they are even using it.

In general the recommendations are sound or at least introduce a higher level of rigor than we have had lately (with the possible exception of John Ellson who is more clueful about software maintenance processes).

In the medium-long term, I think we should attempt to put CVEs fixed into the release notes too. Some people treat it as a best practice (e.g. https://github.com/coreinfrastructure/best-practices-badge/blob/master/doc/criteria.md) and I think it would be nice. That said, I’m not the one fixing CVEs so my opinion shouldn’t count for much :slight_smile:

Seems reasonable, though so far I’ve found it a bit challenging. Taking graphviz#1517 as a case study, AFAICT this is a rough timeline of events:

  1. 2019-03-26: Loginsoft Security Research creates #1517
  2. 2019-04-08: MITRE grants Loginsoft CVE-2019-11023
  3. 2019-04-08: Loginsoft post an advisory about this
  4. 2019-04-09: @scnorth fixes this in commit 839085f8026afd6f6920a0c31ad2a9d880d97932
  5. 2019-04-10: Loginsoft notifies MITRE of their post and the fix
  6. 2019-04-24: Jaroslav Škarvada back ports the fix to Fedora. I assume Jaroslav had also pulled this into the RHEL ecosystem as he has a RHEL affiliation.
  7. 2019-05-03: Fedora announce the issue and its fix in Fedora 30
  8. 2019-05-22: Fedora announce the fix back ported to Fedora 29
  9. 2019-05-22: SUSE announce the issue and its fix in OpenSUSE 15.1
  10. 2019-05-28: SUSE announce the fix back ported to OpenSUSE 15.0

All of this is inherently uncertain because I don’t know what timezone some of the timestamps are in relation to. However, if I had to guess I would say Stephen likely had no idea this had even been filed as a CVE at the time he made the fix (feel free to contradict me, Stephen). Loginsoft make no mention of it on the Gitlab issue.

My point here is that, as long as we are not actively requesting CVEs ourselves, we are never going to be able to tell when we fix one. A third party can even retroactively request a CVE for something we fixed in a previous release. Large software orgs like Google, purveyor of our friendly Google Autofuzz bot, serve as their own CVE Numbering Authority. They are assigned a block of CVEs and often have allocated one to an internal issue they are tracking but have not yet broadcast the allocation.

tl;dr: I think we can do best-effort, but the changelog is never going to be authoritative with respect to CVEs.

I think usually security shops should tell us of the CVE. It’s a shame that didn’t happen here, but I’m still glad they reported it. I wonder if there’s some way for us to subscribe to CVEs being reported for graphviz. As you say, we can start by including the ones we do know about.

Sure, but we have no way of enforcing this.

That I don’t know. It’s possible to search for existing Graphviz CVEs, but some of these I could map back to a specific change or open issue.

Surely it would also be responsible to keep track of which CVEs might still be open.

1 Like

I think we need to change our decision on this. The specific example I previously gave of the ps2pdf change that didn’t deserve a change log entry has caused a lot of disruption to users (e.g. #1763, homebrew-core!57132, linuxbrew-core!20812, …). It’s becoming clear to me that we have a significant group of users who are building Graphviz from source, but also not monitoring Graphviz commits.

I propose we reverse our stance on changes that only affect people building from source and do include changelog entries for these.

1 Like

Perhaps in a separate section or otherwise distinguishable from “normal” usage?