v3.0

v3 Prologue

V3 of LAMS is a big change that brings on board many of the underlying LookML Parser’s output options that have accumulated over time: whitespace awareness, extension/refinement support, a simplified file representation, and imported project support.

The broad overhaul that this required for built-in rules also presented an opportunty. Many of the built-in rules have been made more streamlined and consistent. But, most importantly as of 3.0, built-in rules will be opt-in rather than opt-out. These two changes will greatly improve the ability to publish new built-in rules and patch existing built-in rules without unduly burdening existing LAMS users.

As a result, users upgrading to 3.0 should expect to paste a simple declaration into their manifest to opt-in to each built-in rule.

Beyond that rote change, the primary impact will be to LAMS users who have implemented custom rules, who may have to update their rules for the updated LookML representations.

Usage of LAMS that only uses the built-in style guide rules should not see a large impact, except that some rules may now work correctly based on extensions/refinements, whereas they may have thrown “false positive” errors previously because the parser did not apply any extensions/refinements.

Highlighted changes

• Extensions/refinements awareness: LAMS now supports the ability to lint over logical declarations that are the result of extends/refinements, rather than only lexical declarations. All existing built-in rules except K3 have been updated to check extended/refined objects. In addition, custom rules can take advantage of this - since extension/refinements work at the model scope, these representations can be found under the model construct. The file construct continues to provide the lexical / “as written” representation.
• Whitespace/comment awareness available: Rules may now elect to evaluate whitespace by looking at the $strings property within each object. An initial built-in rule pertaining to whitespce, W1, is also included in the initial release as an example.
• File representation simplification: Files were previously represented as an array. The ordering was not particularly significant. The lack of a meaningful key made the location returned in custom rules opaque and necessitated workarounds. Now, files are represented by an object, keyed by the filepath.

Changelist

• Parser updates
  • $strings whitespace data is now available
  • Parsed file representation is now “by name” (instead of array)
  • Extension/refinement support
  • Support for imported projects (must be locally cloned before invoking LAMS)
• Built-in rules are now opt-in
  • Add #LAMS declarations, like rule: F1 {} to opt-in to a rule. A sample declaration to opt-in to all rules is provided in the readme.
  • Although rules named like [A-Z][0-9]+ are nominally reserved for built-in rules, if you’ve written a custom rule whose name overlaps with a built-in rule, you can mark the rule as custom: yes to disambiguate.
• Built-in rule updates
  • F1: Rule F1 previously evaluated over view declarations in view files. It now evaluates over assembled views in models.
  • F2: Rule F2 previously evaluated over view declarations in view files. It now evaluates over assembled views in models.
  • F3: Rule F3 previously evaluated over view declarations in view files. It now evaluates over assembled views in models.
  • F4: Rule F4 previously evaluated over view declarations in view files. It now evaluates over assembled views in models.
  • K1: Rule K1 previously evaluated over view declarations in view files. It now evaluates over assembled views in models. It now also contains logic that was previously identified as K2.
  • K2: Rule identifier K2 is deprecated, and its rule logic is applied under K1.
  • K4: Rule K4 previously evaluated over view declarations in view files. It now evaluates over assembled views in models.
  • T1: Rule T1 previously evaluated over view declarations in view files. It now evaluates over assembled views in models.
  • T2-T10: These were stated as separate rules in the style guide, but evaluated by one rule in the linter. The are being bundled up as one rule, T2. You may still apply exemptions for individual sub-rules if you wish, though in some cases, dependencies between the sub-rules means that not all parts of the rule will always be evaluated.
  • W1: Rule W1 is added as an initial rule demonstrating the ability to operate on whitespace.
• JSON Path library update
  • The JSON Path library has been migrated from jsonpath, which was rarely maintained, to jsonpath-plus. This is primarily to address the old library’s failure to correctly match union expressions, which were a common use case for expressions like $.model.*['dimension','measure'].*
  • Javascript expressions within path components are strictly limited to a single existence check or equality check against a literal
• Minor bugfixes
  • Fixes issue 123 where exemptions declared directly on the matched object were ignored by custom rules

Migration Guidelines

For any error not detailed below, please submit an issue!

• Opt-in to any built-in rules that you want! See example in LAMS’ Readme
• If you are providing any custom rules that iterate over file matches
  • Change your match from $.files.* to $.file.*. Note that the path/location will now contain the filename.
• If you have newly-erroring views that were previously unchecked due to their being declared in a non-view file
  • You may wish to update the view to comply with the style guide, or apply exemptions for the relevant rule(s)
• If you have opted out of rules T2-T10
  • These rules are now all bundled under rule T2. For a global exemption, you may simply omit rule T2 from your manifest. For local exemptions, or exemptions to the subrules, you may still exempt these individually, although the ID’s of the rules have been updated. (For example, rule “T3” is now “T2.1”)

v3.1

This update introduces, as beta features, centrally managed exemptions, and an add-exemptions output mode. Together, these features serve to enable incremental linting use cases.

Feedback on these beta features and the overall incremental linting use case is welcome in issue #142.

• Centrally-managed exemptions: (BETA) In addition to locally specified exemptions, LAMS can now also use exemptions in lams-exemptions.ndjson in a newline-delimited JSON format.
• Add exemptions as output: (BETA) output=add-exemptions can append any errors from the current run as new exemptions in the lams-exemptions.ndjson file.
• New rules available. Make sure to add them to your manifest.lkml to opt-in. An updated sample snippet to opt-in to all rules is provided on the main README page.
  • K7 - Provide one primary_key. K7 details
  • K8 - primary_key must be or use PK-named dimension(s). K8 details
  • E6 - foreign_key-based joins are many_to_one. E6 details
  • E7 - Explore labels are 25 characters max. E7 details

Patches: - v3.1.1 - Change usage of Node-native readline module for greater Node version compatibility - v3.1.2 - Fix license metadata to consistently reflect MIT license - Update vulnerable dependencies (both in root package and tools/rule-sandbox) - v3.1.3 - Fix rule F1 logic to check all references when there are multiple in a field (Issue 167 PR 168)