|
date. While here bl3ify. Changes are:
Covered-Related NEWS
====================
* 04/11/2004
Release covered-20040210 made. A lot of work has gone into this release to make the report
output more readable and concise. Several bug fixes have been made as well. A GUI is on
the way for report viewing that will be available in alpha version in the next development
release. Below are some of the highlights of this release.
- Added GUI interfacing functions in preparation of upcoming GUI report viewing utility.
- Added more information about expressions to line and combinational logic coverage verbose
information. Rather than just outputting the RHS of the expression, the LHS and assignment
operator (blocking or non-blocking) or IF statement are output to give the user a better
context of the missed logic.
- Fixed bug in param.c where parameters found in the RHS of expressions that were part of
statements being removed were not being properly removed.
- Fixed bug in sim.c where expressions in tree above conditional operator were not being
evaluated if conditional expression was not at the top of tree.
- Changed output of logic in combinational logic verbose coverage reporting to (by default)
use the same format (in terms of endline characters) as the logic was found in the source
code.
- Added '-w [<line_width>]' option to report command that causes combinational logic to be
output to report as much logic as will fit in the value of <line_width> in the report. A
default value of line width is specified internally in Covered to be 105 characters;
however, the user may make this value larger or smaller to suit. This value reverses the
effect of the above bulletin. Added this option to Covered's regression suite to test.
- Completely modified output format of missing combinational logic coverage. Removed a lot
of coverage information that was extraneous. When three or more subexpressions are ANDed,
ORed, logical ANDed, or logical ORed, coverage information is output in a special way to
increase readability/understandability for this coverage.
- Added "GENERAL INFORMATION" section to all reports which specifies general information
about this report (this eliminates a lot of redundant information in the report to improve
readability).
- Added the name of the CDD file from which a report has been generated from in the
GENERAL INFORMATION section of the report.
- When a CDD file is created due to merging CDD files, the names of the original CDD files
are now stored in the merged CDD file. This information is output in the GENERAL
INFORMATION section of the report (created from this merged CDD file) to indicate to the
user this information.
- If a CDD file is created due to merging CDD files and the leading hierarchies in each of
those CDD files are different, a bullet in the GENERAL INFORMATION specifies this and
reminds the user that the leading hierarchy information will not be output in the rest of
the report (instead the string "<NA>" replaces the leading hierarchy information). This
will help to eliminate confusion when viewing the reports and fixes an outstanding bug
in Covered.
- Added starting and ending line information to module structure for GUI purposes.
- Removed scope information in CDD file for expressions, signals and statements. This
information was not used, caused CDD files to become excessive in size and mildly speeds
up reading in CDD files.
- Fixed bugs in combinational logic report section where summary coverage numbers and verbose
coverage numbers did not agree.
- Removed 'c' directory in 'diags' directory and cleaned up Makefile to run regressions.
- Masked off the value of the SET bit in expressions output to CDD files. This information
is not needed and sometimes caused regression failures due to CDD file mismatches on
different platforms or using different simulators.
- Modified regression Makefile to specify the 'vvp' command prior to the compiled VVP
executable when running Icarus Verilog regressions (due to recent change to IV).
- Changed instance-based reports to not merge child instance coverage information into parent
instance coverage information. This is not done in module-based reports, makes reading
this information confusing and doesn't provide us any extra information.
- Fixed bug where modules were being reported in verbose reports when coverage numbers were
100% covered.
- Changed toggle coverage report output to output toggle information in hexidecimal format
versus binary format. This keeps the toggle coverage information more succinct/readable.
Added underlines between every 4th hexidecimal value to help user's to discern the bit
position of a toggle bit.
- Changed the format of the report entirely to enhance readability (many changes here that
the user will immediately see).
- Updated user documentation for new changes and added new section called "Reading the
Report" which will walk the user through several reports and how to interpret the report
information. This section is still in progress at this time.
- Updates to development documentation.
- Lots of new diagnostics added to regression suite. We now have over 200 diagnostics in this
regression.
Special note: Please note that the CDD file format for this release has changed from previous
CDD files and is therefore incompatible with older versions. If you try to read a CDD file
generated from an older version of Covered with the newer version, Covered will tell you that
this cannot be done due to incompatible CDD versions.
* 11/16/2003
Release covered-20031116 made. This development release contains a new way to specify FSMs
within the design by using inline Verilog-2001 attribute syntax. There are also a lot of
bug fixes contained in this release as well as the usual user and development documentation
enhancements. See the list below for more details on the changes made for this release.
- Added better VCD parsing capability to allow bit selects to be "attached" to the signal
names in the VCD variable definition section. The newer versions of Icarus Verilog now
output this format style.
- Added ability to specify FSM location and transition information using Verilog-2001
attributes. Added many diagnostics to regress suite to verify this capability.
- Fixed bug found in stable release that caused an incorrect calculation of unary operations
performed on single-bit values. Fixes bug 835366.
- Fixed bug found in using constant values in the right-hand side of repetitive concatenation
operators. Fixes bug 832730.
- Fixed bugs in reporting of FSM coverage information in the report command.
- Fixed bug in FSM variable binding stage that caused incorrect coverage numbers to be reported
for FSM coverage.
- Fixed bug in handling variables that are too long (more than the allowed 1024 bits). Removes
memory corruption problems when this occurs. Displays warning to user that it has found
a variable that it cannot handle and gracefully disregards any logic that uses these variables.
- Updated user documentation to include new chapter on inline attributes that Covered can
now handle.
- Updated development documentation for new functions added in this release.
* 10/19/2003
Release covered-20031019 made. Lots of modifications to existing structures and supporting
code to increase scoring speed. In my testbenches, I am seeing about a 3-4x improvement
in speed. Additionally, code enhancement for allowing bit selects and signal concatenations
in command-line FSM variable descriptions are now allowed. User documentation has been
updated for these changes. Some bug fixes are also included in this release. The
following list shows the changes from last development release.
- Added ability to parse more complex state "variables". This includes the ability
to specify single and multi-bit signals and the ability to concatenate more than
one signal (or signal bit select) to make a state variable. Please see user
documentation for more information on this.
- Minor tweaks to report format for displaying filenames (only basename of filename
is output instead of the entire path).
- Fixing bug in VCD parser to allow bit select parsing of a variable when the
variable name and bit-select information are not separated by spaces (this is something
that newer versions of Icarus Verilog now does in its VCD files).
- Changed structure for vectors from ints to chars. Each vector element stores information
for one 4-state bit value and its coverage information (instead of storing 4 4-state
variables and coverage information). This reduces memory needed and increases calculation
speed on vectors.
- Fixing bug in signal_from_string function.
- Fixing bug in arc.c related to non-zero LSBs of signals.
- Added new parameter to info line in CDD file that specifies the format of the CDD file.
This is used by Covered to keep CDD files with different formats from being merged, read,
etc.
- Removed LSB information from vector and storing this information in the signal
structure. Reduces memory required, enhances speed, and fixes existing bugs with bit
selects.
- Added more diagnostics to regression suite to test new functionality.
- Updated user documentation for new changes.
* 09/25/2003
Release covered-20030925 made. This release contains the first working FSM code
coverage portion in Covered. There is a lot more to work on in the FSM code coverage
area in the way of automatic FSM extraction and state transition specification, but
this version is able to extract FSM coverage information for an FSM that is located
by the user. Please see user's manual for this release for more details on specifying
FSM location. Summary and verbose reporting are available for FSM coverage at this
point. Additionally, the data format for FSM coverage information in the CDD file
has been finalized. FSM coverage merging is also supported in this release. User and
development documentation has been updated. Please give this development version a
go to get any bugs out of the FSM code coverage engine.
In addition to the FSM coverage support, a bug was fixed in the vector_to_int()
function when converting a vector whose LSB is a non-zero value.
Here is what is on the horizon for FSM coverage that you should expect to see in the
coming development releases.
- Ability for user to specify the location of an FSM using $attribute function.
- Ability for user to specify all possible state transitions for a given FSM on the
command-line and using inline $attribute functions.
- Automatic FSM extraction including locating an FSM and extracting all possible
state transitions.
* 08/20/2003 - Stable Release 0.2.1
Some bugs were found in the covered-0.2 release that needed to be fixed to consider
Covered to be completely stable. This release (covered-0.2.1) contains these fixes
which are outlined below. Please get a hold of this stable release if you have already
downloaded covered-0.2.
- Fixing bug with the initialization of the new symtable structure. Only 255 of the
256 children of each node were being initialized correctly. Fixes a segfault problem with
the symtable_dealloc routine.
- Fixed memory leak problem with file list in parser. This was a long outstanding problem
that has now been understood and fixed.
- Added fclose() after the VCD parsing was complete.
- Fixed a memory problem with the symtable structure that caused other data structure
values to be corrupted.
- Fixed assertion error problem with VCD symbol aliasing.
* 08/16/2003
Stable release covered-0.2 finally made! This release will be the springboard
for adding FSM coverage code, code optimizations as well as a few new features
that should make the score command run much faster. Some important bug fixes
were made in this release and code optimizations have been added to the score
command. If you are getting coverage for a larger design, you should definitely
notice the speed increase. In one of my designs, the speedup was a facter of
a bit more than 3x. The following are the list of changes made for this release.
- Added -ts option to score command to allow the user to see where in the
simulation process the score command is currently at. Please see user
documentation for more details on this new option.
- Fixed bug with multiple wait event statements within same always block.
This means that the CDD files created with the last version of Covered
will be incompatible with the new CDD files.
- Fixed bug with posedge, negedge and anyedge expressions when more than
one of these is found in the same always block.
- Fixed bug in vector comparison function. Vectors will now compare to a value
of true if the values of two vectors (whose bit size is different) are equal
up to the smallest MSB of the two vectors. Before, if two vectors were not
of equal size, a compare would always evaluate to FALSE.
- Removed unnecessary global variables.
- Removed generated development documentation from release and opted to
generate these with a user 'make' in the doc directory (makes release size
smaller and is unnecessary for most users anyways).
- Development documentation updated.
- User documentation and man file updated.
* 08/06/2003
Release covered-0.2pre3 made. It has been quite a while since a release has been made
which has been due to a particularly tricky bug that was found with non-blocking
assignments. This bug generated bad coverage information (this is considered very bad!)
This release contains bug fixes and development documentation updates. If no more problems
are found with this release, I will get the 0.2 release made very soon. The following
is a list of the changes for this release.
- Fixes to line.c and toggle.c to provide better cross-platform support.
- Lots of updates to the development documentation.
- Fixed bug with properly handling hierarchical references in expressions.
- Fixing bug with single-bit parameter handling (caused a diagnostic miscompare
between Linux and Irix OS's).
- Fixed non-blocking assignment bug. This bug affected the order of execution in
Covered's simulator which resulted in bad coverage information being generated.
- Fixed bugs in divide, mod, left shift, right shift and some other expression types
to avoid converting variables that have unknown values to integers (which results
in Covered errors at run-time).
The list of changes is short, but the changes made are very necessary to getting reliable
coverage numbers from Covered. Please get a copy of this version and test it out so that
we can get the stable 0.2 release made ASAP.
* 02/18/2003
Release covered-0.2pre2 made. Bug fixes and enhancements for allowing more Verilog
code to be parsed without spewing parsing errors. The list of open bugs is empty at
the moment. I will be working on enhancing the user documents and development
documents in preparation for the stable release. The stable release will be made
next unless there are new bugs found for which the bug fixes convince me that additional
testing is necessary. The following is a list of the changes for this release.
- Fixed bug with copying instance trees for instances of modules that were previously
parsed and built into the main instance tree.
- Fixing bug in file finder so that only missing modules are displayed after the
parsing phase is completed.
- Updated output of filenames as they are parsed to give more consistent look.
- Fixing bug with leftover tmp* file when missing module error is reported.
- Adding parsing support for pullup, pulldown and gate types though these are not
supported for coverage at the current time (probably will be supported after stable
release).
- Adding parsing support for real numbers in statement delays.
- Fixing case where statement is found to be unsupported in middle of statement tree.
The entire statement tree is removed from consideration for simulation.
- Added preliminary support for parsing attributes though the parsing support is not
complete at this time.
- Fixing bug with line ordering where case statement lines were not being output
to reports.
- Fixing bug with statement deallocation for NULL statements within statement trees.
- Updates to parser for new bison version 1.875
- Added support for named blocks
- Fixing bug with handling of preprocessor directives with leading whitespace.
- Fixes/optimizations to db_add_statement function which avoids stack overflow errors.
- Added check in regard to -i option to score command. Bad -i values would cause no
coverage information to be generated but would not tell user explicitly. Error
message now provided with -i option is not specified but is needed and/or -i option
is incorrect.
- Lots of parser updates to be able to parse UDPs, escaped identifiers, specify blocks,
and some other various Verilog code that was causing parse errors or assertion errors.
- Fixed proper handling of the event type.
- Fixed bug with merging constant/parameter vector values which caused assertion error
in report command when reporting on a merged file.
- Fixed user error message for merge command when CDD files are unable to be read.
- Added new type to CDD for general CDD file information. This allows CDD files from
different testbenches with the same DUT instantiated to be properly merged.
- Fixed problem with generating report from CDD file that has not been scored. Covered
detects that the CDD file has not been scored and outputs an error message to the user
not allowing them to generate these reports.
- Added support for reading bit selects from VCD files (this information was previously
ignored). This was necessary as other simulators bit blast module ports in VCD files.
- Updated look of instance reports to display full hierarchy of an instance instead of
the instance name and the instance name of the parent module. Much easier to locate
the instance in the design now.
- Fixed bug with using -D/-Q option with merge command.
- Added merge regression testing capability to regression suite.
- Updated build environment for RedHat 8.0 requirements.
- Updates to regression suite
- Development documentation updates.
The way that Covered looks and feels for 0.2 stable release is set in stone now. Please
make sure that you test this version as much as possible to get any leftover bugs out of
the code. I only plan on updating documentation, adding code comments, and fixing bugs.
If any bugs are sent in, a 0.2pre3 release will be created, otherwise, I will make
the 0.2 stable release available.
I've got some exciting things in Covered's future in plan after 0.2 stable release,
including FSM support, new text report look, code optimizations, support to be fully
Verilog-2001 compliant, a parallel scoring algorithm, and a new GTK+ interface.
Happy testing!
* 01/05/2003
Happy New Year! Release covered-0.2pre1_20030105 made. This release is primarily a
bug fix release; however, many of the bugs required larger changes than would be
expected before a stable release. The most significant change being to the lexer which
is now split into a preprocessor and a normal lexer (before these two functions were
combined into one). As such, this is the first prelease with a second release most
likely expected before stable release. The following is a list of changes included in
this release.
- Preprocessor split out from lexer to allow proper handling of defined values within
code.
- Added -p option to score command to allow user override of preprocessor intermediate
output file.
- Fixed bug where report output was not squelched when -Q global option specified on
report command line.
- Modified regression suite to verify CDD file generation (was being performed before),
module report generation (new) and instance report generation (new) to make sure that
report output was consistent.
- Fixed bug where integer, time, real, realtime and memory data types used in expressions
where considered to be implicitly defined and given 1-bit values. When these types
are seen in expressions now, they are ignored by Covered (caused nasty segfault).
- Fixed bug when a parameterized module is instantiated more than once in a design
(sent error message to user when this occurred).
- Fixed bug where a parsed module that was required but not at the head of the module
list was not being found by the parser.
- Added internal assertions and code to verify that we never try to overrun arrays
in the VCD parsing/running stage (caused nasty segmentation fault).
- Reorganized code for symtable symbol lookup and value assignment.
- Fixing bug where a parameterized module that was instantiated in a design more than
once was not getting the correct parameter value(s).
- Fixed module search algorithm to reparse a Verilog file that contains a module
that was previously ignored (not needed at the time) but is later found to be
needed.
- Created tree.c and tree.h to handle new module search algorithm and to replace
preprocessor define tree structure.
- Updated development documentation.
- Updates to user manual and manpage for new -p option, notes from this release
and a new section that starts to describe what logic is analyzed by Covered and
which code is not analyzed.
I've got some fairly large designs being run with this version of Covered and the regression
suite has grown to over 130 diagnostics with more on the way. Keep the bug reports coming!
|
|
in recent bulk builds.
Release covered-20021214 made. This release is a bug fix release. See list below
for details. Bugs that lead to infinite looping in the score command and segmentation
faults should now be cleared up. Please let me know if there are any other bugs that
need to be addressed before first stable release. Development documentation updated
to match changes in files. Regression suite has been updated quite a bit from last
time. There are now over 125 diagnostics in the regression suite (my goal was to
write about 100 before first stable release).
- Segmentation fault fixes in report command
- Parser can now handle all net types (not just wire). Diagnostics added to regression
suite to verify their proper handling.
- Parser updated to handle net declaration assignments (e.g., wire a = b & c;).
Diagnostics added to verify proper handling.
- Added human-understandable error messages in parser to help identify file and
line number along with a quasi-helpful error message description.
- When parser error is found, Covered exits after parsing phase without continuing
to write CDD file.
- Fixed bug where a multi-bit select expression existed in a module that was
instantiated more than once. Assertion error fired in this case.
- Updated regression suite for VCS testing.
- Fixed bug where parameters were used in modules that were instantiated more than
once.
- Fixed bug that dealt with parameters (see param6.1.v for test case).
- Fixed bug where a delay statement was the last statement in a statement block used
by Covered. Added diagnostics to verify correct behavior.
- Fixed infinite loop problem with db_add_statement function.
- Fixed infinite loop problem with statement_set_stop function.
- Fixed bug with parsing order. When an instance is found for a module that has
already been parsed, the instance was incorrectly being handled. Bug replicated
with instance6.v diagnostic.
- Fixed output of edge-triggered events to add @(...) around the expression (they
were easily confused with other code that could exist on the same line).
- Fixed bug in parser to not allow module to be parsed more than once.
- Fixed bug that lead to an assertion error (see instance6.1.v for test case).
- Fixing bug with calculating list and concatenation lengths when MBIT_SEL
expressions were included.
- Changed Covered's handling of -y directories. Before, all files in these directories
were fed into the parser to look for missing modules. Now, when a module is needed,
the module name is used to find the matching filename in the -y list (basically,
the -y option works like the -y option in Icarus Verilog and VCS). This fix really
streamlined the parsing phase and fixed several bugs.
- Memory declarations are now properly ignored (produced segmentation fault previously).
- Fixed report command to display all lines and expressions in order according to
their line number (the problem is REALLY fixed now).
- Removed hierarchical references from being scored.
All in all, you should notice a huge improvement in the parsing speed, syntax errors are
reported better, more Verilog syntax should be handled properly, the score command will
run a bit faster than before, and the reports should be a bit easier to read. Segmentation
faults and assertion errors should become lesser in number (if not gone altogether?).
I am feeling pretty confident that we are getting close to a stable release as I have
been able to generate a CDD file for a chip that is millions of gates in size (CDD file
was created in the range of 30 - 45 seconds!) Keep the bug reports coming. I have some
things to work on for next release already.
|
|
This is a development snapshot. Packages of the released/stable
versions will be imported as 'cad/covered' when available.
Covered is a Verilog code coverage analysis tool that can be useful
for determining how well a diagnostic test suite is covering the
design under test. Typically in the design verification work flow, a
design verification engineer will develop a self-checking test suite
to verify design elements/functions specified by a design's
specification document. When the test suite contains all of the tests
required by the design specification, the test writer may be asking
him/herself, "How much logic in the design is actually being
exercised?", "Does my test suite cover all of the logic under test?",
and "Am I done writing tests for the logic?". When the design
verification gets to this point, it is often useful to get some
metrics for determining logic coverage. This is where a code coverage
utility, such as Covered, is very useful.
Please note that this package is a development snapshot and while it
contains the latest and greatest features, it may be buggy as well.
There is a seperate package which is made of the stable releases.
|
