Linting tools don’t get much love from design engineers. The reasons for this are valid. Linting is often an afterthought after verification is essentially complete. The rule set is often the default rules from the vendor or an old unmaintained one that was put together by a junior engineer many years ago. The tools don’t get much attention from vendors and language support is often lacking. All in all linting becomes a tedious process of weeding out irrelevant warnings in the search for that one bug which can make the last weeks of pain and suffering worthwhile.
This is a shame. Linting or static analysis should be a fast and effective bug hunting tool. Lint should be the the second tool to be run on RTL, just after elaboration. I believe that I can find a way to make lint find the simple and stupid bugs that I write in a matter of seconds with a low false positive rate. My personal goal for 2015 is to find that way.
For 2015 I envision that my verification work flow will look like this:
- elaboration
- high impact lint
- Quick formal check of embedded SystemVerilog Assertions
- Personal code review
- UVM testbench and simulation
- Exhaustive lint
When it is time for the personal code review, I will kick off an elaboration in our synthesis tool. Full synthesis and LEC will probably start while developing the UVM testbench. There is no reason to start before this time as buggy RTL often results in completely incorrect synthesis reports when big chunks get optimized away.
By structuring my workflow this way, I believe that I can get the highest possible code quality at the earliest possible moment throughout a project.
High impact lint
High impact lint is normal linting with a personal rule set that actually matters. I am going to take a standard rule set and simply remove rules that I find noisy. I see five levels of warnings that I linter can emit:
- Definitely a bug (Latches, combinatorial loops)
- Almost always a bug
- Could be a bug, I don’t know
- Almost never a bug
- Not a bug, why is this a rule?
Linters are usually faster when the rules are fewer, so I am inclined to keep only the first level, but I am hesitant to leave out level 2. I should leave out level 3 and below, but I am not sure that I successfully will be able to.
I know for sure that the rule set that I will use is for my use only. I don’t expect anyone else to use it. I want full control over the rule set so that I can maintain a high impact from the linting. Other people make different mistakes, have different coding styles and want different rules to apply in order to detect problems. I don’t want to spend time discussing with others whether a specific rule should be present in my rule set.
I don’t know whether I should keep score on the rules. I am inclined to do so, but I don’t know if I will find this too tedious. The alternative is to keep two rule sets, put level 2 and 3 on the candidate rule set and check the candidate set when it is time for the personal code review.
Formal verification
I am not sold on formal verification, yet. Formal verification has its place in base components and module interfaces, but it fails in the messy real world of chip design. Still, if I can verify simple SystemVerilog Assertions that I have embedded into my design before writing a testbench or running a single simulation, why should I not do that? The worst case is that I can’t prove or disprove any of the assertions and I have wasted a few minutes.
I have only a little experience with formal verification, but my initial findings are that I find a bug somewhere with low effort and that I write more assertions to check more things because it is faster that way. It feels a lot like the first time I run simulation with a new testbench, just that the feedback cycle is shorter as I write short assertions instead of testbench code.
Personal code review
Code reviews has been shown to have one of the highest bug detection rates per hour invested. There is no reason why I should not do code reviews. Some structured software development approaches place code review just after compilation, but I don’t think that this is a pragmatic place to put it. I prefer that the code review comes after the simple automatically detectable problems have been fixed. This allows me to focus on logic problems. The personal code review is not as strong as code reviews done by others, but it doesn’t require buy-in from others so that is what I am going with for 2015.
UVM testbench
Constrained random verification (CRV) is the gold standard of functional verification. There is no way that I would sign-off on a design without an UVM testbench with sufficient code coverage and a finished verification plan. Having said that, there are disadvantages to CRV. UVM testbenches with all the checks and tests take a significant amount of time to write and run. I expect CRV to be the tool that takes me to 100%, but I don’t see a reason why I should start with CRV at 0% and debug simple logic bugs with it. I want to hunt for the hard and nasty bugs with UVM, not the simple and stupid ones. If I can be at 50% with high impact lint, quick formal verification and code review when writing the UVM testbench then I will be very happy.
Exhaustive lint
There is no way around it. Someone is going to require that I wade through an endless amount of lint warnings from an ineffective linting rule set at some point. The goal is to push this task until the end of the project where it can do the least harm. The end of a project is heavier on computer cycles than engineering cycles. It is a good time to go through warnings while waiting for tools to complete. Exhaustive lint is also the lowest value task. If schedule pressure demands that something gets cut then the exhaustive lint is the correct task to remove.
Wrapping up
I think that I have found a way to improve my productivity by incorporating linting into my verification work flow. I am starting on a new project in 2015 where I can test my new work flow. I will need to change the way I write RTL to suit the new tools that I am going to use, but I can live with that. I hope that linting first will give higher quality code at an earlier point in time.