Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Great Unit Test Tracker #201

Open
1 task
donn opened this issue Oct 2, 2023 · 1 comment
Open
1 task

The Great Unit Test Tracker #201

donn opened this issue Oct 2, 2023 · 1 comment
Assignees
Labels
✨ enhancement New feature or request

Comments

@donn
Copy link
Member

donn commented Oct 2, 2023

Misc Unit Tests

  • pdk_compat.py
@donn donn changed the title Checklist of Step Unit Tests (excl. OpenROAD) Checklist of Step Unit Tests Oct 10, 2023
@efabless efabless deleted a comment from kareefardi Oct 17, 2023
@donn donn changed the title Checklist of Step Unit Tests Write Unit Tests for All Steps Oct 17, 2023
@kareefardi
Copy link
Collaborator

kareefardi commented Oct 17, 2023

Step Unit Tests

  • General case:
    • success
    • failure:
      • bad input
      • bad configuration. e.g. floorplan impossible util...
    • reproducibles that are fixed
  • Checker.DisconnectedPins
  • Checker.IllegalOverlap
  • Checker.LVS
  • Checker.LintErrors
  • Checker.LintTimingConstructs Checker, Magic Unit Tests #233
    • fail
    • success
    • not found
  • Checker.LintWarnings
  • Checker.MagicDRC
  • Checker.TrDRC
  • Checker.WireLength Checker, Magic Unit Tests #233
    • fail
    • success
    • not found
  • Checker.XOR
  • Checker.YosysChecks
  • Checker.YosysUnmappedCells
    - [ ] KLayout.OpenGUI (xvfb breaks 80 times a second im not testing this)
  • KLayout.Render
    • success comparison with a known png
    • fail - bad DEF.
  • KLayout.StreamOut
    • success no macros
    • success macros only
    • success hybrid (macros + std cells)
    • fail bad DEF.
  • KLayout.XOR
  • Magic.DRC Checker, Magic Unit Tests #233
  • Magic.SpiceExtraction - Skipping this for now. Since there is no obvious way to verify the generated spice file.
    • NOTE: MAGIC_NO_EXT_UNIQUE - JC (user) test case add to integartion tests - a design
    • success using GDS
    • success using DEF
    • success with IllegalOverlap
  • Magic.StreamOut Checker, Magic Unit Tests #233
    • success no macros
    • success macros only
    • success hybrid (macros + std cells)
    • success hybrid - Magic macro std cell source = PDK
    • fail bad GDS
    • fail bad DEF.
  • Magic.WriteLEF Checker, Magic Unit Tests #233
    • Abstract and full
      • def
      • def hybrid
      • GDS
      • GDS hybrid
    • bad input DEF
    • bad input LEF
    • bad input GDS
  • Misc.LoadBaseSDC
  • Netgen.LVS
    • pass
    • pass blackbox
    • pass full
    • fail on each of the following:
      • design__lvs_device_difference__count
      • design__lvs_net_differences__count
      • design__lvs_property_fails__count
      • design__lvs_errors__count
      • design__lvs_unmatched_devices__count
      • design__lvs_unmatched_nets__count
      • design__lvs_unmatched_pins__count
    • fail bad spice
    • fail bad verilog
  • Odb.ApplyDEFTemplate Enhancements + Unit Tests for Pin-Placement Odb Steps  #250
    • success
    • bad input DEF
    • mismatching input DEF
  • Odb.CustomIOPlacement Enhancements + Unit Tests for Pin-Placement Odb Steps  #250
    • success
    • fail due to umatched pins
    • fail due to bad input odb
    • fail due to bad configuration file. There are multiple ways in which the configuration file can be bad.
      TBD point of failures in the configuration file.
  • Odb.DiodesOnPorts: Waiting for Tweak Classic Flow's Antenna Insertion Regime #255
    • success on in
    • success on out
    • success on both
    • skip on none
    • fail on bad DIODE_CELL config - multiple points of failure
    • fail on bad odb
  • Odb.HeuristicDiodeInsertion: Waiting for Tweak Classic Flow's Antenna Insertion Regime #255
    • success on different threshold values
    • fail on bad odb
    • fail on bad DIODE_CELL config - multiple points of failure
  • Odb.ManualMacroPlacement
    • success
    • fail bad odb
    • fail misconfigured MACRO object - potential enhancement ??
    • no need to test cfg file as it is under deprecation.
  • Odb.ReportDisconnectedPins
    • positive disconnected power and signal pins
    • fail bad odb
  • Odb.ReportWireLength
    • success
    • success no wires in odb
    • fail bad odb
  • OpenROAD.BasicMacroPlacement
    • Not implemented ??
  • OpenROAD.CTS
    • CTS is sensitive to SDC
    • There seems to be alot of configuration paramters for CTS. I am not entierly sure yet what would be a strategy to test all of them
    • success multiple clocks
    • success multiple generated clocks
    • success single clock
    • success no clock
    • fail bad odb
  • OpenROAD.CheckAntennas OpenROAD Unit Tests #244
    • success no violations
    • success violations
    • fail no routes
    • fail bad odb
  • OpenROAD.CheckSDCFiles
  • OpenROAD.DetailedPlacement
  • OpenROAD.DetailedRouting OpenROAD Unit Tests #244
    • fail too little iterations
    • fail layer (min) in guide not accessible
    • fail layer (max) in guide not accessible
    • fail pin not accessible
    • fail no access obs
    • success
    • success assert 1 thread
    • success assert 4 thread
    • fail bad odb
  • OpenROAD.FillInsertion OpenROAD Unit Tests #244
    • fail bad filler cell
    • fail success no space
    • success
    • fail bad odb
  • OpenROAD.Floorplan OpenROAD Unit Tests #244
    • success relative
    • success aboslute
    • success 0 margin
    • fail -ve margin
    • fail -ve utilization
    • fail margin > area
    • fail bad aspect ration
    • test insert_tiecells
    • test impossible dimensions - TBD. Two points passed to die area are treated as bbox points. Hence there is no impossible dimension (maybe 0,0,0,0). However a diea area set to "DIE_AREA": "0 100 34.5 57.12" results in a bbox of 0.0 57.12 34.5 100.0. This can be considered a user error and should be captured.
    • fail bad netlist
  • OpenROAD.GeneratePDN
  • OpenROAD.GlobalPlacement OpenROAD Unit Tests #244
    • success
    • success time driven only
    • success routability driven only
    • success skip-initial-place
    • success no timing no routability
    • fail high util
    • fail high padding (results in high util)
  • OpenROAD.GlobalPlacementSkipIO
  • OpenROAD.GlobalRouting
  • OpenROAD.IOPlacement OpenROAD Unit Tests #244
  • OpenROAD.IRDropReport
  • OpenROAD.LayoutSTA
  • OpenROAD.OpenGUI
  • OpenROAD.RCX OpenROAD Unit Tests #244
    • Success simple design
    • Success simple macro
    • Success macro + design level std cells
    • Success no_merge_via_res
      Success is measured by reading in the generated spef file and report unannotated wires.
  • OpenROAD.RepairDesign
  • OpenROAD.RepairDesignPostGPL
  • OpenROAD.RepairDesignPostGRT
  • OpenROAD.ResizerTimingPostCTS
  • OpenROAD.ResizerTimingPostGRT
  • OpenROAD.STAMidPNR
  • OpenROAD.STAPostPNR
  • OpenROAD.STAPrePNR
  • OpenROAD.TapEndcapInsertion OpenROAD Unit Tests #244
    • Success
    • Fail large distance / small area
    • Fail bad master cells
  • Verilator.Lint
  • Yosys.EQY
  • Yosys.JsonHeader
  • Yosys.Synthesis

@donn donn pinned this issue Nov 5, 2023
@donn donn changed the title Write Unit Tests for All Steps The Great Unit Test Tracker Nov 5, 2023
@donn donn added the ✨ enhancement New feature or request label Nov 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✨ enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants