Release Notes

[tabatkins]

(This thread documents the updates in comments; this original comment always reflects the latest version.)

Version 4.2.2

• Fixes the parser to allow for <<foo [<<bar>>]>> shorthands to be parsed,
  now that that's an allowed syntax for CSS grammars.

Version 4.2.3

• Allow <<foo []>> (with an empty arg).

Version 4.2.4

• Didn't realize that argparse (the cli library I use) doesn't correctly handle the same option name being used both at the top-level and in a subparser, so moved the --no-update top-level option to a new internal name. (No effect on user-facing code, except that bikeshed --no-update spec and bikeshed update --skip-manifest both correctly work now`.)
• Fixed Some solution for avoiding the lowercasing #861, so refs with "significant" casing get that casing preserved when they're printed in indexes/etc. (Before, "dfn" type refs were always lowercased, so for example "ECMAScript" would print as "ecmascript" in the indexes.) There's some minor heuristics involved so sentence-starting refs don't preserve their casing accidentally.

Version 4.2.5 - 4.2.7

Insignificant version bumps, just me trying to get py.typed included, so mypy knows about Bikeshed's types.

Version 4.2.8

• Bumped a lot of dependencies.
• Finally fixed the NodeJS favicon used by the CanIUse panels.

[tabatkins]

Version 2.0.0

• Feature: Added darkmode styles!

  This is the reason for the version bump, as if you were adding your own styles to your document, they probably aren't darkmode-aware and will look bad when the readers are in darkmode. If you're just using Bikeshed's own styles, tho, everything will work exactly as before.

• Feature: Added new conditional-inclusion feature, allowing you to remove sections of a spec (typically in boilerplate) based on the document's Status. If you have more things you want to base a conditional inclusion off of, let me know!

  (Currently, the processing order of this happens after all the 'datablocks' (pre with special class that triggers Bikeshed handling) happens, so even if those are inside of a conditional that gets excluded, they'll still get processed. But everything else happens after conditional stuff is removed.)

• Bugfix: The rest of IDL's built-in types are now autolinked.

[tabatkins]

Version 2.1.0

• Feature enhancement: Conditional inclusion can now conditionalize on the presence of a custom boilerplate element (something with boilerplate=foo attribute) or the presence of a custom text macro, so you can include a chunk of related HTML only when the thing it's trying to refer to actually exists!

• Feature: Autolink conjugations (the thing that lets you write [=applying=] when the original definition was just <dfn>apply</dfn> now apply to the first word of a phrase as well as the last. This will usually catch the verb in a multi-word phrase, like "apply the widget", so you can write [=applying the widget=] and have it Just Work. (But all the conjugations apply, so noun-headed terms like "widget controlled by the user" can be autolinked with [=widgets controlled by the user=] now.)

• Feature: Biblio autolinks (like [[FOO]]) can now be switched into "inline" display mode, where rather than displaying as "[FOO]" and linking to the bibliography, they display as the actual document's title and link straight to them. (They still get added to the index.) You can turn it on individually with [[FOO inline]], or set the default for all biblios in the spec with Default Biblio Display (and switch them back to "index" mode with [[FOO index]]).

• Bugfix: Better darkmode styles! Various fixes thruout, including to certain boilerplates.

• Bugfix: bumped the versions for several dependencies to the most recent form, just for general dependency-hygiene reasons

• Bugfix: If you used Editor Term to rename what the spec-metadata section called "Editors", various small side-effects that should specially trigger for the Editor lines... wouldn't. They do now.

[tabatkins]

Version 2.3.0

Whoops, didn't write release notes for the 2.2 versions! Oh well!

• Feature: Global -a flag to force Bikeshed to emit ASCII-only in all its messages.

• Feature: When using WPT Path Prefix with a lot of tests missing, don't spam the full list out, just log a warning that there are a bunch.

• Feature: Biblio autolinks can now take the obsolete keyword to allow an intentionally-obsolete reference (otherwise Bikeshed complains and automatically switches your ref to what obsoleted it).

• Feature: Finally got daily test updates working, so we run with the most modern set of specs rather than ones that are obsolete by months.

• Bugfix: Stop ignoring certain test types from WPT, so any test can be linked.

• Bugfix: Track what data files exist locally manually, rather than relying on the list matching what the remote data last contained.

• Bugfix: Grab the correct set of boilerplates for the specified Status.

• Misc: Lots and lots of fixes to make Bikeshed lint-clean.

[tabatkins]

Version 2.3.1

• A bunch of dependency bumping
• More importantly, Automatic Image Size Detection, care of @frivoal! If your <img> elements don't already have width/height attributes on them, Bikeshed will do its best to auto-detect what the sizes are, and insert them into your markup automatically. This is a small but nice perf improvement for specs, avoiding layout-jiggle as images load in.

Version 2.4.0

• Added the NOTE-ED, NOTE-WD, and NOTE-FPWD statuses, for handling the new somewhat-more-official "Note track" for W3C documents, so you can have boilerplates with appropriate verbiage to indicate it's a Note-track document. (REGISTRY-* statuses will come in a bit, when the updates to W3C Process finish up enough to allow publishing such a document.)
• Minor fixes to boilerplates and styles. Shouldn't be anything you notice, but if you're tracking diffs in your generated output you'll see some chatter.
• Another banger from @frivoal, <wpt> elements have some new features:
  • <wpt hidden> processes the element but omits it from the output (useful for leveled specs that need to include all the tests in their folder so they can benefit from WPT Prefix, but don't actually have a place to put the tests for up/down-level versions)
  • <wpt title> lets you specify a descriptive paragraph for the tests in the block, formatting it just above the test list in the output
  • Multiple adjacent <wpt> elements will now be merged in the output
  • Best of all, rendered <wpt> elements are now <details> elements, and the new WPT Display: closed renders them closed by default (the previous inline value and its new alias open render them open by default, looking approximately the same as they previously did). The default styling for a closed <wpt> is a tiny out-of-the-way "TESTS", suitable for displaying even in live, published specs. Play with it for now, consider adding it to your defaults.include file. I plan to flip this to be the default in a near-future version.

[tabatkins]

Version 2.4.1

• Several small boilerplate tweaks to some groups
• <wpt title> attribute is now parsed as Bikeshed source

[tabatkins]

Version 2.4.2

• A few boilerplate updates, as always
• Properly strip the internal-use attributes from <pre class=include-code> elements from the output document, so they won't cause your document to fail HTML validation.

Version 2.4.3

• Boilerplates again.
• A couple of widlparser dependency updates

[tabatkins]

lol been skipping release notes for a while!

Version 2.4.4

• boilerplates again

Version 2.4.5

• boilerplates
• removed some ancient (and now incorrect) lines from the linking-defaults file

Version 2.4.6

• accidentally did the previous release on Windows and picked up Windows line endings

Version 2.4.7

• fixed the url link-defaults was pulled from during update!
• boilerplates
• bumped a lot of dependencies
• fixed 3d037a4, which caused text to get dropped in some circumstances

Version 3.0.0

• New major version! Nothing exciting, I just pushed a breaking change for security reasons; you now have to explicitly opt-in, from the command line, to allowing Bikeshed to grab data from outside the input file's directory, or running arbitrary code via some extensions features.
• biblio entries and links now use cite, such semantic e4a654b
• boilerplates

Version 3.0.1

• boilerplates
• actually lint the 3.0.0 code that restricted use of files outside the source directory

Version 3.0.2

• boilerplates
• some correctness fiddling to the 3.0.0 code

Version 3.0.3

• ??? I did literally nothing, weird.

Version 3.0.4

• boilerplates (jeez I'm getting tired of this)

Version 3.0.5

• boilerplates

Version 3.0.6

• boilerplates

Version 3.1.0

• NO MORE BOILERPLATES UPDATES. Bikeshed now stores and versions its boilerplates over at https://github.com/tabatkins/bikeshed-boilerplate, and brings in updates via bikeshed update like any other data files.

Version 3.2.0

• fix a lot of lint errors from 3.1.0
• propdef tables now link their "Applies To" header 980486a
• Issues lists (from bikeshed issues-list) can now have intro text c2f0309
• Format links to GH issues better 8bb5020
• Added AOM as a group and megagroup, plus some new statuses for them

Version 3.3.0

Big new WPT features - inline test results!

• <wpt> blocks now contain pass/fail information in the main browsers! f07e1bf
• wpt lines indicate when a test file contains multiple tests, and how many
• If there's a WPT path prefix (or I can infer one), add the corresponding http://wpt.fyi link in the spec-metadata section up top
• also add pass/fail data for all spec's tests combined to the spec-metadata section

And some other random stuff:

• Stop doing the weird "implicitly call .format() on messages" and just use f-strings or explicit .format() everywhere 368df1c
• Update the dockerfile 24054d8
• propdef tables now link their "Canonical Order" header ed3425a
• bikeshed template now outputs a Status that actually works, so you can use its output immediately, as intended.

Version 3.3.1

• adapt to the Process 2021 changes that got sprung on me without announcement

Version 3.3.2

• I'd accidentally been omitting the wpt data files from the distributed version. They'll be as out of data by default as all the other built-in data files, but they should at least exist. Fixed now. bbc3648

Version 3.3.3

• remove some now-unused W3C NOTE statuses 6cf9175
• Only set the LOGO macro if Logo metadata is actually set 31a2f03
• Allow conditional logic to look at non-custom metadata too; combined with previous, you can now add a default logo if a spec doesn't set their own logo. 31a2f03

Version 3.3.4

• CanIUse blocks are now closed by default, like MDN blocks be0e7d0
• Fix update link for boilerplate files a892fb1
• Actually clean up old boilerplate files that no longer exist cace53f

Version 3.4.0

• Add pngwg to W3C groups (I need to move this data out of this repo, too.....) c9308f3
• New "direct" biblio style that still formats the link text as the shortname (to match the biblio entry), but skips the biblio and links straight to the document. 1f16618
• Documented the W3C's spec-prod GH action, which makes it easy to auto-publish with Bikeshed
• Disallow merged Privacy/Security sections, per new W3C rules 893894c
• When you've just got a few headings missing their ID, output them individually so they get line numbers c29faa6
• Document how to set "html metadata" (attributes on html, meta/link/style/script elements) 8eaccd9
• Detect and scream when you're likely hitting the dumb years-old argparser bug about spaces in the values of unknown flags beb1f28
• Strip the common WS prefix from fenced code blocks, same as I do for pre elements 53a8100
• Add a new debug command to output all the metadata passed to the document, from all the possible sources cd66b7b

[tabatkins]

Version 3.4.13.4.2

• Previously, Bikeshed would helpfully not spam you with all the missing tests that matched your WPT Path Prefix if there were more than a handful, since there were legit reasons to have a bunch of untracked tests (tests not for your level, maybe you're in the middle of annotating, etc). Now that we have <wpt hidden> to do a well-targeted silencing when required, that's no longer a problem, so let'er rip. f965859

• If you've only got empty/hidden wpt elements, Bikeshed no longer adds the JS, spec-metadata entries, etc. Style still comes thru so the blocks render, at least. 4d354f1

• The legacy processor Bikeshed replaced handled links to properties and descriptors identically; 'font-family' would gladly link to the property or the @font-face descriptor, depending on what was in the spec. That never worked well in the first place (the Fonts spec would randomly link to each), and while Bikeshed could at least detect the issue and complain about it, it's annoying to deal with when properties are usually what you wanted to link to. It's gotten worse as we've added more descriptors whose names collide with properties, like 'width' or 'aspect-ratio'.

  So, propdesc autolinks without an explicit for value are now always treated as property autolinks; 'width' will always link to the 'width' property, and never ask if you meant the 'width' MQ or CQ descriptor. You can still use the syntax with a for value, like '@media/width', to link to descriptors. This might cause some new link failures, but more likely it'll cause some existing link failures to go away and link correctly. f9584d2

(I accidentally forgot to pull on my release box before I cut the 3.4.1 release, so 3.4.2 actually contains all the stuff.)

[tabatkins]

Version 3.5.0

• Fixed the automatic indent-removal for pre blocks in some cases bd9a15d
• Added the [:http header:] autolink syntax
• Added the "permission" definition type f092c3a
• Trust anchor-block entries, even if the spec you're linking to is marked as obsoleted in SpecRef b1c6bfa
• Allow div around dt/dd, including markdown-generated ones, since that's valid in HTML. Previously it would often kinda work by accident if you contorted your markup in a certain way, but it was real bad. 790e008
• Lots of adventures in beginning add typing information to Bikeshed. Long way to go yet.
• Properly autolink the type signatures in argumentdef blocks. ac0d420
• Added a few W3C groups and statuses

Version 3.5.1

• Added the fedidcg group

Version 3.5.2

• Made section autolinks also trigger a biblio entry, which I believe is the last place that should be able to do so. Also localized the code that tracks this so all the entry points do the exact same thing; this means some of your biblio entries might change casing since they're consistent across all callsites now. There's still some mismatch between what specs I know about via biblio data and what specs I know about from definition data, which I know will cause at least a few instances where "the same" biblio entry shows up twice under slightly different names, but that's a separate issue that won't be fixed until I move Bikeshed's definitions entirely to Reffy. fdd5936

[tabatkins]

Version 3.6

• <div assert> now works, to get the same auto-generated ID as <assert> does, but for block-level. (<assert> turns itself into a <span> in the output.) (Thanks, @dj2!)
• Fixed some new duplication of normative/informative refs caused by the section autolinks fixes from 3.5.2.

Version 3.7

• Can now build multi-file specs directly from an (uncompressed) TAR file! This is worthless locally, but you can use this to send multi-file specs to the API server. (Thanks, @kainino0x!)

[tabatkins]

Been a while since my last batch of release notes!

Version 3.7.1

• WPT tests mentioned multipled times in a spec now properly only count once for the test counts (Only count each WPT test once for the summary view #2304)
• WPT summaries use the proper Stable channel browser logos (WPT: Use Stable Channel Browser Logos #2296)
• Started some serious mypy-ification

Version 3.7.2

• Fix a bug I caused while mypy-ing.
• More mypy-ification.

Version 3.7.3

• Fix a minor bug from an argument being removed in py 3.9.
• More mypy-ification. (Seeing a trend? This continues until October.)

Version 3.7.4

• Just mypy stuff. Unsure why I cut this version, honestly?

Version 3.7.5

• Fix the merging of adjacent MDN blocks.
• Fix caniuse and MDN darkmode styles.
• Made the markdown em/strong replacers work better. They're still not right, but they're now less wrong.
• Complain About: broken-links yes now has a timeout so it can't hang indefinitely.
• Fixed a cache accidentally getting shared across documents, continually growing when you generate a bunch of specs (like bikeshed test does...)
• Fixed how heading data is recognized from the data source, to better handle how ReSpec formats its markup.
• FINISHED MYPY-IFICATION

Version 3.7.6

• Properly add some new non-py files introduced by 3.7.5 to the release data.

Version 3.7.7

• Fixed a spot where if you force-generate (so die() doesn't stop the program) it would end up with inconsistent state and cause a Python error anyway.

Version 3.8.0

• Added simple def tables, formatted like propdef/etc but arbitrary keys
• Added bikeshed update --force-manifest that forces it to do a manifest update even if the remote's date indicates your local data should be more recent.

Version 3.8.1

• Fix anchor recognition, which was broken in 3.7.5 by the heading fix.

Version 3.8.2

• Added a missing from __future__ import annotations that was causing a crash in older Python versions.
• Only warn about missing WPT tests under your specified prefix once, rather than twice.

Version 3.9.0

• Properly flag if your anchor-block refs have a local URL (which breaks other stuff anyway).
• Fix a long-standing issue where section autolinks could generate a similar or duplicate biblio entry to what was already generated by definition autolinks.
• Obsolete biblio links are now LINKERROR, not FATAL.
• Moved the self-links from notes/issues/examples to avoid being cut off by their 'overflow' value.
• HTTP Header autolinks now use the correct typography per Infra.
• Added Complain About: mixed-indents yes to catch when you mix tabs and spaces in bad ways.

[tabatkins]

New big batch of release notes:

Version 3.10.0

• Added explicit support for "Logical Property Group" in propdef blocks

Version 3.10.1

• Biblio links now have the reference's title as their title (unless they're inline, since those already have the title as their text)
• Better parsing for Markdown URL syntax. (Think it's still not perfect, but substantially improved wrt parsing the url portion.)
• Fixed the bikeshed echidna --just-tar option to actually echo the TAR file, rather than whatever broken nonsense it had ended up doing.
• Fixed a bug with negative ranges in the CSS <length [-100px, 100px]> syntax, and just switched to using hyphen-minus rather than the fancy math minus.

Version 3.11.0

• Switched Bikeshed's ref database to using the W3C's WebRef service, rather than plinss's Shepherd service. Lots of commits related to this.
• Note markers (the "Note:" bit) now have class=marker, as the docs said they should.

Version 3.11.1

• Reworked the "unused var" error-checking so I could output line numbers, to help find the error.

Version 3.11.2 & 3.11.3

• Bikeshed moved official ownership! Now a project under the Specification Editors CG in the W3C, at https://github.com/speced/bikeshed.

Version 3.11.4

• After some poking at WebRef to get the data complete, Bikeshed no longer relies on Shepherd at all.
• Put a really cute little progress bar on both the manual update info, and test running.

Version 3.11.5

• Fixed the "mixed indentation" lint to properly allow tabs followed by spaces. (Spoilers: it's still broken at this point.)
• Added the first part of my new bespoke HTML parser.
• Fiddled with the dfn panel generation/positioning for better a11y.

Version 3.11.6

• Fixed the "mixed indentation" lint correctly.
• Try to infer the indent from source analysis, rather than just looking at the first indented line.

Version 3.11.7

• Fiddle with ID generation for links that dfn panels need to link to, to make them more likely to be unique.

Version 3.11.8

• Fix how I package bikeshed for release, so I'll stop forgetting to include new .css and .js files when I add them.

Version 3.11.9

• Fixed a filepath.

Version 3.11.10

• Removed a good chunk of the ARIA stuff I tried to add, since it technically violated the ARIA spec (even tho I think it was right). Grumble, grumble.

Version 3.11.11

• Unify a lot of the CanIUse/MDN panel code, particularly positioning. Switch them from doing some funky abspos-with-static-position stuff to being positioned purely with JS, since they can trivially overlap otherwise.
• Specs marked as "fork" in WebRef are no longer included in the db.
• A bunch of fiddling with the update code to untangle the old mess of transforms needed for mixing the Shepherd and WebRef sources. Since it's all WebRef now I can make it much simpler.

Version 3.11.12

• Accidental rerelease, lol

Version 3.11.13

• Include .bsfont files in the build too. Surprised this is the first time someone's run into this issue!

Version 3.11.14

• Preserve the start-tag's attributes when handling all the various datablocks (propdef, railroad, include, etc).
• Move the semver file into the bikeshed folder so it's actually installed by pip. Turns out the top-level files aren't!

Version 3.11.15

• Nothing, just didn't notice the semver got bumped again while I was fiddling with the release script.

Version 3.11.16

• Since 3.11.14 started using the new bespoke HTML parser, it can accidentally surface some HTML mistakes that were previously silently accepted. The error messages weren't showing the right line tho - now fixed.

[tabatkins]

Version 3.11.17

Turns out using just a little of a new HTML parser causes problems when my old hacky line-based code is, well, doing things line-by-line, and thus only feeding the parser part of a tag when a tag is split across multiple lines. So now I just use the HTML parser for the entire pre-Markdown parsing pipeline. (Then Markdown runs, generates some HTML, and feeds its result straight to the lxml parser. Both of these will be rewritten in time as well.)

It looks like this doesn't cause any issues with the testsuite, and hopefully should resolve the spurious fatal errors that 3.11.16 was throwing. Lmk if you see any issues!

[tabatkins]

Version 3.11.18 - 3.11.21

• Me increasingly realize I'd fucked up my testing, and trying to fix all the nits that were breaking people's builds.
• Eventually giving up, pushing the problematic code to a branch to bake more, and reverting back to before 3.11.14.

Post-Mortem

So, 3.11.14 caused some problems. I was throwing my new HTML parser at my datablocks.py code, so I could reliably parse the start tag and preserve all of its attributes. Problem is that the datablocks code was processing the spec line-by-line, so if you had a multi-line tag (such as, for example, having a newline in an attribute value), the parser would complain that it hit EOF without the tag being closed. This didn't have an effect on any of my tests, because nobody had a multiline tag for one of their datablocks, so it didn't actually matter for the spec itself. It did matter that it was suddenly throwing spurious fatal errors, however, as that breaks people's builds. (First mistake.)

At this point I could have reverted the code and worked on it more in a branch, and should have. (Second mistake.) Instead I dug in - my new code was nicer, and I wanted to replace all my hacky text munging anyway, so I might as well just rip it out entirely and use my new parser.

This took a few days, but it was productive work, and I was happy with it. I did this via a big chain of "wip" commits, with test rebasing going on arbitrarily in commits, so when I thought I was done, I did a big squash commit - git reset --soft to the first commit, commit all the code, rebuild tests and verify, commit them, done. The tests looked good! Surprisingly few tests changed, and those that did were almost entirely catching errors, which was great! Problem is I did the squash wrong - I needed to reset to the commit before my first in the chain. Due to my mistake, I didn't see all the test changes my code had made, only the changes my commits after the first commit made. So I ended up missing a ton of problems that would have been fairly obvious. (Third mistake.)

Takeaway Fixes

• First, I'm not tracking build messages, just build results. I've broken people's builds before due to this, which is annoying. Also, the message output would have shown some of the errors more clearly than the HTML changes did, mixed as they were with some trivial formatting changes. I'm going to take on a Q2 project to fix this, and start tracking build messages as part of the test suite.
• Second, when I see breakage, unless the fix really is trivial I should go ahead and eagerly revert. I knew the fix wasn't going to be trivial here, but wanted to make forward progress, and made the wrong call. In the future, always revert eagerly if I can't immediately fix the error.
• Third, the commit issue was just a silly avoidable mistake. To ensure I don't do this again, whenever I do this kind of significant work in a branch, I'm going to only merge it in as a PR in GitHub, rather than locally, so there's another layer of visual check that I'm getting what I expected.

[tabatkins]

Version 3.11.22

• Significant refactoring of the "dfn panels" that show up on definitions and list all the links in the document. To reduce the amount of repetitive HTML sent down the wire, these are now sent in a reduced JSON form and constructed in JS on load.

• Added a new Ignore MDN Failures metadata, allowing you to silence an MDN failure (to deal with transient errors due to something moving between specs and MDN temporarily being wrong about where a feature lives).

• You can use <wpt skip-existence-check> to make the WPT block, well, skip the existence check. This lets you use URLs that aren't in WPT at all, or aren't there yet, and still get a similar-looking test block. (No pass/fail data or the helper links like source vs live, but otherwise same as normal.)

Version 3.11.23

• Fixed a bug in highlighting, where sometimes a <pre class=include-code> would drop the last line of the include. (<pre class=include-code> ignores the last line if there's no newline when line numbers are required #2521)

• Slightly improved how I auto-generate IDs for some links (that need to be linked to by dfn panels, etc.) If you were depending on those auto-genned IDs, uh, they're different now. (Using a custom anchor with an empty fragment from multiple sections gives a "Multiple elements have the same ID" error #2531)

Version 3.12.0

• Refactored all of my darkmode handling to work properly. There's a new Darkmode metadata you can use to turn off Bikeshed's darkmode styles.

Version 3.12.1

• Removed some final remnants of the old CSSWG testsuite data. If you don't know what that is, don't worry it didn't affect you.

Version 3.13.0

• Removed the ABSTRACTATTR macro, now that nobody consumes it. (It was the same as the ABSTRACT macro, but escaped to be suitable for use in an attribute value. But ABSTRACT is HTML content so it was weird to do that in the first place.) This almost certainly doesn't affect you either.

• Removed flake8 from my linting pipeline, added ruff to replace it, and turned on a FUCKTON of lints. It can't quite replace pylint yet, but it's close, and it's SO FAST.

• If you do something that makes an invalid spec (like invoking Bikeshed without an input file, with no .bs file in the current directory), I now properly catch that and fail gracefully from several commands rather than dying with a Python stacktrace.

• Did a bunch of refactoring of the Install/Use section of the docs to emphasize using pipx to install and manage Bikeshed, as a much easier and safer install tool, and to expand some examples of how to use Bikeshed within GitHub CI.

• Bikeshed now checks if your datafiles are relatively old and automatically attempts to update them, so you don't need to run bikeshed update yourself. I did a lot of refactoring in this code and it seems to be great, but please let me know if there are any issues.

[tabatkins]

Version 3.14.0

Let's try that HTML parser rewrite again! Coming at it more gradually now; I do one early pass over the document with the new parser and then just reserialize back to a string. I haven't yet replaced the datablock, markdown, or shorthand parsing, and I still pass it to the pre-existing HTML parser to actually build a tree, so mostly this is just some extra work being done, but those'll all be absorbed into this work in later phases.

As I say in #2602, this brings a few benefits immediately:

• Detects some markup mistakes, like duplicated attributes.
• Correctly adds line numbers to everything in your source. (Some elements coming from metadata or datablocks still won't have correct line numbers.)
• Handles markdown code spans, code blocks, CSS <<type>> autolinks, and CSS ''maybe'' autolinks correctly, in the parser, rather than hackily via a regex.

It then still runs the datablock, markdown, and finally the existing HTML parser over the spec, so this is probably slightly slower at the moment, but those will be eaten by the parser in later phases.

Known issues with existing specs' markup:

If you were working around the hacky ` parsing by using \` inside of a code block
(which shouldn't have any parsing done inside)
it no longer does any parsing inside.
Fix: Remove the spare \ so you just have valid JS again.
If you weren't working around this your spec was probably broken,
and now it's fixed!

The HTML parser runs before the Markdown parser right now
(fully integrating the two together is the next project),
so a tag broken across a line inside a blockquote
will parse incorrectly
(it's closed prematurely by the blockquote's > at the start of the next line)
Fix: just put the whole tag on one line for now.

Previously, ''<foo>'' would make a maybe autolink to <foo> as a value.
Now it's equivalent to <css>&amp;lt;foo></css>,
which is broken,
but arguably it was always broken in the first place.
Fix: change to ''<foo>''.

The Markdown behavior of "if you have spaces at both the start and end of a code span, remove one from each side" is now properly implemented. A few specs relied on it removing any amount of any whitespace, so now there's an extra space sometimes if you did a linebreak inside your code span for some reason.
Fix: Put the code span all on one line, or at least don't linebreak between the end of the content and the closing ticks.
(The serializer still collapses starting/ending whitespace down to a single character to make the content look better, so that might still be doing more stripping than you want too.)

[tabatkins]

A few small updates in the last few weeks.

Version 3.14.1

• The AB is now a W3C group (I really need to move this data to bikeshed-data rather than in the Python...)
• (temporarily) allowed CR as whitespace, to fix a confusing parse error that some people were getting.

Version 3.14.2

• Made = in unquoted attribute values not an HTML parse error, so you can still have unquoted URLs with query strings in your attributes. (This'll be partially reverted in a future version, as the parse error apparently legitimately catches a lot of problems in the wild where people accidentally omit the space between two unquoted attributes, but I'll keep the exception for attributes known to contain URLs, like href and src.)

Version 3.14.3

• reverted the CR change from 3.14.1, instead properly normalized linebreaks when the source file comes from a URL. Turns out requests doesn't do this for you, while the rest of Python file loading does.

Version 3.14.4

• Added a Die On metadata, paralleling the --die-on command-line flag, that controls what level of error message results in the spec failing to build entirely. No change to existing documents (it still defaults to "fatal"), and the command-line flag still overrides anything coming from metadata.
• Allowed W3C Community Groups to use the w3c/UD status, since it's an unofficial status anyway, and useful to distinguish from their "real" draft statuses.

[tabatkins]

Version 3.14.5

• I now finally track console output as part of the testsuite as well. Previously I just monitored the actual spec output, but new fatal errors could show up without changing the output (and even changes to warnings could break builds that were using --die-on=warning to ensure a clean build). Shouldn't run into surprises like that anymore.
• When using <pre class=simpledef>, I no longer automatically sentence-case the keys, so you can do things like {{someMethod()}}: describing the method and the link will no longer get broken.

[tabatkins]

Version 4.0.0

Woo, major version bump! Content-wise this isn't a break-worthy update, but Bikeshed has officially deprecated support for Python 3.7 (and 3.8), and now requires a minimum version of 3.9. As far as I can tell, this is the highest widely-supported version among default installs now; the old OSes that bundled 3.7 by default are all now out of long-term support.

• The link-checking lint now properly handles a timeout.
• If you're building a spec from within a github repo, the commit is now recorded in <meta name=revision>, to accord with the registered meta name for this purpose.
• Some minor fixed to how dfn panels work.
  • Clicking the "self-link" in the panel no longer triggers it to move down to the lower left corner; that only happens if you click one of the references to the dfn.
  • Dfn panels no longer get clipped by overflow on ancestors of the dfn. (This did require switching to positioning them more manually with JS, rather than being able to rely on CSS's "static position" concept; please let me know if anything looks wrong.)
  • Dfn panels now more intelligently avoid overflowing the screen, sliding nicely away from the edge to stay visible.
• In addition to dfn panels, we now have link panels, which show on hover and fade away more eagerly. They give a few useful bits of information about the link now, and will be developed more in the future.
• Added a new --die-when command-line argument, letting you toggle between a fatal error stopping Bikeshed immediately, or just stopping it at the end from actually producing output. Defaults to "late" now (continues processing, only dies at the end), so you can see all your fatal errors at once. You can pass --die-when=early if you have too many errors and want to fix them one-by-one.
• Both --die-on and --die-when can now be specified in your metadata (including in your group's defaults), as Die On and Die When, taking the same values as the command-line args.
• The manifest for Bikeshed's data files is now written in KDL rather than a bespoke format, so it should be more easily changeable in the future. (This shouldn't affect your authoring experience at all.)
• This existed before, but it's now documented: if, for whatever reason, you need to skip Bikeshed's auto-update check it does at startup to ensure its data files are up-to-date (for example, if you're on a plane and dont' want to wait for the network to time out every time), you can pass --no-update on the command line.
• The [TITLE] macro, used to pipe the document's title into the <title> element, now omits markup in your Title metadata and just contains text. (<title> is specially parsed in HTML, similar to <xmp>.) This means you don't necessarily need to write both Title and H1 metadata if you want to put some markup into your document's <h1>. (But if you're using markup with semantic visuals, like <sup> or <sub>, you probably do still want to do something different there.) This ended up fixing accidental markup getting spewed into the document title for several specs in my testsuite.
• Syntax highlighting no longer strips whitespace from the start and end of the block. Spot-checking a few places where this caused a minor output difference, it doesn't look like it actually causes a visual difference. But this does fix a bug that a few specs hit when combining highlighting with existing markup that subtly corrupted the text.
• If you had <pre highlight=foo><code>...</code></pre>, previously it would get double highlighted. (Shouldn't have been visible, given the highlight styling most people are using, but it was silly verbose markup.) No more!
• Fixed a crash bug if you tried to use an HTML character reference that either expanded into two codepoints or a single astral codepoint. (Nobody hit that in my testsuite, but still.)

Most of my time was spent on making Bikeshed's new HTML parser more powerful and more correct (mostly in 5c6f7af). This shouldn't have any big effect on you (some caveats, below), but every step forward I make here makes me so much happier.

• All of the early text-munging I previously did before running markdown/html parsers is now moved into the HTML parser: macros, curly apostrophes, and line-ending em dashes. This means Bikeshed will no longer accidentally modify things inside of script/style/xmp elements, markdown code spans or blocks, or Bikeshed "datablocks" (the <pre class=foo> things that are specially handled).
  • A few specs in my testsuite had manually corrected for this, using Markdown escapes so their JS template strings weren't detected as Markdown code spans. These'll need to be removed; the \ is now output as part of your spec. Sorry. (However, more specs hadn't corrected for it and just had broken output; they're now automatically fixed.)
  • Macros are now valid only (a) in text, or (b) in attribute contents. Nothing in the testsuite used them anywhere else, so this should be fine, but FYI.
  • Macros can now correctly handle having more macros in their content. I'll catch accidentally-recursive macros so you won't hang Bikeshed with infinite replacements, tho.
• Several metadata lines that are meant to contain HTML are now fed to Bikeshed's new HTML parser as well, so you'll catch errors there too.
• So are your boilerplate files (so the new macro parser can do its job). Again, this might catch some HTML errors in your boilerplate; open a PR in speced/bikeshed-boilerplate if
• A stupid amount of fiddly work went into making line numbers in error messages more common and more useful. Line numbers from things outside your core document now give their context, so you won't be confused looking at line 20 of your spec when it's actually an error at line 20 of your header.include file, etc.
• I also fixed a bunch of off-by-one errors in line numbering. If you still see any errors that misattribute their line number, please let me know.
  • Exception: line numbers like "Line ~20" (with the tilde) still happen for the remaining linking shorthands that don't know their original line number; they're instead reporting the line number of the nearest construct (HTML element or Markdown block) that does know its line number. If that's incorrect, tho, let me know.
  • Exception: If an error like the previous bullet point occurs in top-level raw text that doesn't get wrapped in a Markdown paragraph, it might report the line number of the wrapper element coming from your header.include. This sucks, I know.
  • Both of these will be fixed in an upcoming version before EOY, when I migrate all the stuff that gets parsed after Markdown/HTML parsing into the bespoke parser, so they'll know what their actual source line number is.
• HTML comments are handled slightly differently internally now; this should generally be undetectable (or work better!) except for perhaps some trailing whitespace appearing where it wasn't before (but was present in your markup). Please let me know if something looks to have messed up, tho.

I did a lot of careful test review to make sure things are as correct as possible. Aside from the bits I listed above, and the possibility of some new markup errors being flagged,

[tabatkins]

Version 4.0.1

• Tiny fix - I had some invalid CSS in a new feature that was causing validators to complain.

[tabatkins]

Version 4.0.2

• Fixed some cases where an <l> element wasn't turned into a valid HTML element.

Version 4.0.3

• Totally refactored my JS/CSS inclusion code. Just an internal thing, tho scripts/styles will move around a bit since they're now alphabetized by their boilerplate name, and the couple that host JSON data changed how they store it.
• Rewrote several of the JS scripts that were unintentionally spending a lot of computation at page load. For large specs like DOM, this changes ~2s of computation to ~100ms.

Neither of these should actually affect specs in a user-observable fashion; let me know if anything seems to be newly wrong.

[tabatkins]

Version 4.1.0

Several miscellaneous fixes; I wanna get these out before I dive back into the parser.

• Now that dfnpanels contain linking guidance, they're useful on all dfns, so Bikeshed now generates them for all dfns. (Previously, since they only contained links to where they were referenced, they were only generated for dfns that had local references.)
• The pass/fail colors for MDN panels was accidentally still inline, so they've been pulled out into a color var so they can be darkmoded.
• If you write <dfn><<length [0, 10px]>></dfn> (defining any type shorthand with a range), Bikeshed will now auto-infer that it is actually a value definition, rather than a type definition. This might change some IDs, if your dfns weren't tagged before and you were relying on auto-detection.
• I now properly handle Python errors raised during metadata parsing (rather than just exploding).
• When you turn off (with the Boilerplate metadata) a script or style with an associated style/script, I now properly omit the associated stuff too, without you having to manually turn them off.
• Updated the WebIDL Parser version, to capture some recent changes.
• Bikeshed's auto-update feature that keeps your datafiles up-to-date now does a fast check for internet access, so you're not stuck waiting for a timeout when you're, say, on an airplane.
• Also, auto-update will no longer fall back to manual update if the manifest update fails; it's supposed to be a fast best-effort thing, and manual updating is SLOWWWWWW.
• Also, if manual updating throws an error, I now properly handle that.
• The Big Text functionality invoked by bikeshed source is completely rewritten.
  • Most importantly, it now keeps the original comment text in the document, so you can just fiddle with it inline and re-run the command, rather than having to delete the generated comment and add back a new <!-- Big Text: foo --> comment.
  • Created a second, narrower font file, too, and switched to it by default. You can fit more letters on screen now!
  • Font files are now written in KDL, rather than a bespoke format.

[tabatkins]

Version 4.1.1

A few small fixes.

• Before, I only fetched one sub-path of WPT data. I used WPT Path Prefix if it was present, or else tried to infer a common prefix from the existing tests. (And in either case, would reject the path if it was the root, so I didn't pull all of WPT in.) Tests outside that path would work, but wouldn't get pass/fail piecharts and wouldn't contribute to the pass/fail boxes at the top of the spec. Bikeshed now handles multiple paths, and infers them from the provided tests automatically.
• The link-popups added a bit ago had some busted listeners for auto-closing them, so they'd close if you hovered a link in them or clicked anywhere in them (preventing you from highlighting, for example). Fixed now.

[tabatkins]

Version 4.1.3

• Sorted the WPT path fragments from the previous version, so the output is stable rather than reliant on set iteration order.
• Removed the code in dfn popups that prevented the page from scrolling when the target was already somewhere on screen, as it also prevented the hash from updating.
• Firefox has a buggy implementation of .offsetTop on some internal table elements, and that made dfn/link popups misalign when their anchor was inside of a table with a caption. Switched to using getBoundingClientRect() to calculate their positions instead, which consistently works everywhere.

[tabatkins]

Version 4.1.4

• The --die-on and --die-when flags are now properly piped into the metadata infrastructure, so they'll win over less powerful Die On/When metadata as they were intended to.
• HTML comments in def blocks are now properly ignored, as they were before the parsing switchover when I was unsafely stripping them from everywhere.

[tabatkins]

Version 4.1.5

Tiny release.

• Link panels now trigger on mousedown rather than mouseover so they're not as annoying when you mouse across the page. To keep them up without following the link, you click on them, pull your mouse down into the panel, then release.

Version 4.1.6

• Extended attributes were accidentally being put into the data-type attributes in IDL attributes, making attributedef tables break. No longer! (Thanks, @tidoust)
• The var-coloring code in algos accidentally regressed when I rewrote into JS, and was only normalizing the first whitespace in a var name. Now normalizes all of them.
• Unless you've opted out with Dark Mode: no, Bikeshed now automatically adds a <meta name=color-scheme> to your doc indicating it's darkmode-aware. If you're publishing a W3C spec (detected by "has a W3C stylesheet"), it also inserts the W3C darkmode stylesheet, which provides darkmode colors.
• Linking scripts (<script src=...>) are now indented like normal in your source. This is nothing, it was just bothering me.
• bikeshed watch and bikeshed serve weren't properly overriding the Die On metadata, so they'd stop watching when they hit a fatal error. Fixed!

[tabatkins]

Version 4.1.7 & 4.1.8

(4.1.7 was an accidental no-op release.) Tiny update!

• Added htmlwg as a recognized W3C Group
• Updated widlparser

[tabatkins]

Whoops, left a few version off the changelist. They're all minor except the last.

Version 4.1.9

• Just adding a new WG (SOLID CG) to the data files.

Version 4.1.10

• Made the var-highlighting colors (the backgrounds that show on up <var>s inside of <div algorithm> blocks when you click the vars) darkmode-aware.

Version 4.1.11

• Made the default tracking-vector image darkmode-aware.

Version 4.1.12

• Just adding a new group (W3T) to the datafiles.

Version 4.2.0

• Moved the data about what groups and statuses exist, which statuses a group is allowed to use, etc. over to a datafile in the bikeshed-boilerplate repo. Previously, updating this requiring rolling a new Bikeshed version; now it can be updated alongside new boilerplates being added! No more silly updates like 4.1.9 and 4.1.12.

• If you're missing some IDs that the MDN data expects to be present in your spec, the error message is a little better now.

• Fixed a weird corner case: as long as there's only a single overload for the method, you can always autolink a method without passing any args: {{Foo/bar()}} links to the bar() method, regardless of how many args bar() actually takes.

  Because IDL methods and attributes occupy the same namespace, you were also allowed to autolink to methods without using parens at all, like {{Foo/bar}} to link to the Foo.bar() method. However, due to some slight mismatches in how two different systems interfaced, this only worked if the method could be called without arguments. If there were required arguments, it didn't work; but {{Foo/bar()}} (parens, no args) still worked fine.

  This is fixed now, tho there are a few specs in the end-to-end testing database which now have ambiguous autolinks as a result. It's likely a few specs in the wild might run into the problem as well.

• Fixed a crash bug that could occur while I was figuring out your spec's indent level, if the spec contained at least 50 space-indented lines but they were all indented with only a single space.

[tabatkins]

Version 4.2.1

• Fixed a couple of W3C URLs 61fea1b

• Made bikeshed spec pay attention to --print=json (previously, only bikeshed refs used it), and document that value. 296e6fd

[tabatkins]

Version 4.2.2

• Fixes the parser to allow for <<foo [<<bar>>]>> shorthands to be parsed,
  now that that's an allowed syntax for CSS grammars.

Version 4.2.3

• Allow <<foo []>> (with an empty arg).

Version 4.2.4

• Didn't realize that argparse (the cli library I use) doesn't correctly handle the same option name being used both at the top-level and in a subparser, so moved the --no-update top-level option to a new internal name. (No effect on user-facing code, except that bikeshed --no-update spec and bikeshed update --skip-manifest both correctly work now`.)
• Fixed Some solution for avoiding the lowercasing #861, so refs with "significant" casing get that casing preserved when they're printed in indexes/etc. (Before, "dfn" type refs were always lowercased, so for example "ECMAScript" would print as "ecmascript" in the indexes.) There's some minor heuristics involved so sentence-starting refs don't preserve their casing accidentally.

Version 4.2.5 - 4.2.7

Insignificant version bumps, just me trying to get py.typed included, so mypy knows about Bikeshed's types.

Version 4.2.8

• Bumped a lot of dependencies.
• Finally fixed the NodeJS favicon used by the CanIUse panels.