{"id":662,"date":"2017-09-17T22:38:23","date_gmt":"2017-09-17T22:38:23","guid":{"rendered":"http:\/\/www.vidarholen.net\/contents\/blog\/?p=662"},"modified":"2017-09-17T22:38:23","modified_gmt":"2017-09-17T22:38:23","slug":"shellcheck-and-shadowed-case-branches","status":"publish","type":"post","link":"https:\/\/www.vidarholen.net\/contents\/blog\/?p=662","title":{"rendered":"ShellCheck and shadowed case branches"},"content":{"rendered":"

As of the the latest<\/a> commit, ShellCheck<\/a> will try to detect shadowed case branches.<\/p>\n

Here’s an adaptation from an unnamed script on GitHub: <\/p>\n

\r\ncase $1 in\r\n    -h|--help)\r\n        help\r\n        exit 0\r\n        ;;\r\n    -h|--hub)\r\n        hub=$2\r\n        shift\r\n        ;;\r\n    *)\r\n        die \"Unknown option: $1\"\r\n        ;;\r\nesac\r\n<\/pre>\n
The original case<\/code> statement was significantly longer, so you’d be excused for not noticing the problem: -h<\/code> is used for two different branches. Because of this, -h<\/code> as a short option for --hub<\/code> will not work. <\/p>\n
If you run ShellCheck<\/a> on this example now, you will get a pair of helpful warnings:<\/p>\n
\r\nLine 4:\r\n    -h|--help)\r\n    ^-- SC2221: This pattern always overrides a later one.\r\n \r\nLine 8:\r\n    -h|--hub)\r\n    ^-- SC2222: This pattern never matches because of a previous pattern.\r\n<\/pre>\n
Very simple and probably somewhat useful in certain cases, right? Well, it gets slightly more interesting.<\/p>\n
Here is another example adapted from the wild:<\/p>\n
\r\ncase $1 in\r\n    -h|--help|-?)\r\n        usage\r\n        exit\r\n        ;;\r\n    -v|--verbose)\r\n        verbose=1\r\n        ;;\r\n    *)\r\n        die \"Unknown option: $1\"\r\n        ;;\r\nesac\r\n<\/pre>\n
Did you spot the same problem? ShellCheck did:<\/p>\n
\r\nLine 4:\r\n    -h|--help|-?)\r\n              ^-- SC2221: This pattern always overrides a later one.\r\n<\/pre>\n
Since an unescaped ?<\/code> matches any character, it will match also match -v<\/code>, so the short form of --verbose<\/code> will not work.<\/p>\n
Similarly, it recognizes two separate issues in this example:<\/p>\n
\r\n    -*|--*) die \"Invalid option: $1\" ;;\r\n    --) shift; break ;;\r\n<\/pre>\n
The end-of-option --<\/code> marker will never be recognized, and -*|--*<\/code> is redundant because the first already covers the second.<\/p>\n
These are all very simple cases, but this also works more generally. Here’s a fabricated music sorting script where the bug would be exceedingly hard to spot in a longer list of bands:<\/p>\n
\r\ncase \"${filename,,}\" in\r\n  *\"abba\"*.mp3 ) rm \"$filename\" ;;\r\n  *\"black\"*\"sabbath\"*.mp3 ) mv \"$filename\" \"Music\/Metal\" ;;\r\nesac\r\n<\/pre>\n
So how does it work?<\/p>\n
There are very clever ways of determining whether one regular language is a superset of another by intersecting it with the complement of the other, and checking the result for satisfiability. <\/p>\n
ShellCheck uses none of them.<\/p>\n
I’ve written a regex inverter<\/a> before, and that level of complexity was not something I wanted to introduce.<\/p>\n
Instead, ShellCheck’s pattern intersection and superset supports only basic DOS style wildcard patterns: ?<\/code>, *<\/code> and literals. It just does a simple recursive match on the two patterns.<\/p>\n
Let’s call the patterns A and B, and we wish to check if A is a superset of B, i.e. if A matches everything that B does. <\/p>\n
We have two arbitrary shell patterns that we want to turn into a simplified form, while ensuring we don’t simplify away any details that will cause a false positive. ShellCheck does this in two ways:<\/p>\n
It creates A in such a way that it’s guaranteed to match a (non-strict) subset<\/a> of the actual glob. This just means giving up on any pattern that uses features we don’t explicitly recognize. $(cmd)foo@(ab|c)<\/code> is rejected, while *foo*<\/code> is allowed.<\/p>\n
It then creates B to guarantee that it matches a (non-strict) superset<\/a> of the actual glob. This is done by replacing anything we don’t support with a *<\/code>. $(cmd)foo@(ab|c)<\/code> just becomes *foo*<\/code>.<\/p>\n
Now we can just match the two<\/a> patterns against each other with an inefficient but simple recursive matcher. Matching two patterns is slightly trickier than matching a pattern against a string, but it’s still a first year level CS exercise.<\/p>\n
It just involves breaking down the patterns by prefix, and matching until you reach a trivial base case:<\/p>\n