Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling FPGA device support in PDDF #13477

Merged
merged 5 commits into from
Mar 15, 2023

Conversation

FuzailBrcm
Copy link
Contributor

@FuzailBrcm FuzailBrcm commented Jan 23, 2023

Why I did it

To enable FPGA support in PDDF.

How I did it

  1. Added FPGAI2C and FPGAPCI in the build path for the PDDF debian package
  2. Added the support for FPGA access APIs in the drivers of fan, xcvr, led etc.
  3. Added the FPGA device creation support in PDDF utils and parsers

How to verify it

These changes can be verified on some platform using such FPGAs. For testing purpose, we took Dell S5232f platform and brought it up using PDDF. In doing so, FPGA devices are created using PDDF and optics eeproms were accessed using common FPGA drivers. Below are some of the logs.

[ logs will be attached below ]

root@sonic:/home/admin#
root@sonic:/home/admin# show ver |more

SONiC Software Version: SONiC.master.0-dirty-20230111.010655
Distribution: Debian 11.6
Kernel: 5.10.0-18-2-amd64
Build commit: 3176b15ae
Build date: Wed Jan 11 09:12:54 UTC 2023
..

Platform: x86_64-dellemc_s5232f_c3538-r0
HwSKU: DellEMC-S5232f-C32
ASIC: broadcom
ASIC Count: 1
Serial Number: CN01WJVTCES0006A0050
Model Number: 01WJVT
Hardware Revision: N/A
Uptime: 04:18:55 up 4 min,  1 user,  load average: 0.62, 0.62, 0.30
Date: Wed 25 Jan 2023 04:18:55

Docker images:
REPOSITORY                    TAG                              IMAGE ID       SI
ZE
docker-orchagent              latest                           63262c7468d7   38
5MB
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# show platform summary
Platform: x86_64-dellemc_s5232f_c3538-r0
HwSKU: DellEMC-S5232f-C32
ASIC: broadcom
ASIC Count: 1
Serial Number: CN01WJVTCES0006A0050
Model Number: 01WJVT
Hardware Revision: N/A
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# lsmod |grep pddf
pddf_fpgapci_module    16384  0
pddf_xilinx_device_7021_algo    16384  0
pddf_xcvr_driver_module    32768  0
pddf_fpgapci_driver    32768  3 pddf_xilinx_device_7021_algo,pddf_fpgapci_module,pddf_xcvr_driver_module
pddf_xcvr_module       20480  0
pddf_led_module        28672  0
pddf_fpgai2c_driver    20480  1 pddf_led_module
pddf_fpgai2c_module    16384  1 pddf_fpgai2c_driver
pddf_mux_module        16384  0
pddf_cpld_driver       20480  2 pddf_xcvr_driver_module,pddf_led_module
pddf_cpld_module       16384  1 pddf_cpld_driver
pddf_client_module     16384  7 pddf_fpgai2c_module,pddf_fpgapci_module,pddf_xcvr_driver_module,pddf_led_module,pddf_xcvr_module,pddf_cpld_module,pddf_mux_module
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# systemctl status pddf-platform-init.service
● pddf-platform-init.service - PDDF module and device initialization service
     Loaded: loaded (/etc/systemd/system/pddf-platform-init.service; enabled; v>
     Active: active (exited) since Wed 2023-01-25 04:14:39 UTC; 8min ago
   Main PID: 304 (code=exited, status=0/SUCCESS)
      Tasks: 0 (limit: 19107)
     Memory: 0B
     CGroup: /system.slice/pddf-platform-init.service

Jan 25 04:14:32 sonic python[304]: :- parseDatabaseConfig: Sonic database confi>
Jan 25 04:14:39 sonic pddf_util.py[304]: Checking system....
Jan 25 04:14:39 sonic pddf_util.py[304]: PDDF has no PDDF driver installed....
Jan 25 04:14:39 sonic pddf_util.py[304]: Installing ...
Jan 25 04:14:39 sonic pddf_util.py[304]: System rebooted in PDDF mode, hence ke>
Jan 25 04:14:39 sonic pddf_util.py[304]: Creating devices ...
Jan 25 04:14:39 sonic systemd[1]: Finished PDDF module and device initializatio>
Warning: journal has been rotated since unit was started, output may be incompl>
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# pddf_psuutil mfrinfo
PSU    Status    Manufacturer ID    Model      Serial          Fan Airflow Direction
-----  --------  -----------------  ---------  --------------  -----------------------
PSU1   OK        DELL               02RPHXA00  CNDED0003R07ZP  Unknown
PSU2   NOT OK    DELL               02RPHXA00  CNDED0003R07YF  Unknown
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# pddf_psuutil seninfo
PSU    Status      Output Voltage (V)    Output Current (A)    Output Power (W)    Temperature1 (C)    Fan1 Speed (RPM)
-----  --------  --------------------  --------------------  ------------------  ------------------  ------------------
PSU1   OK                       12.40                  8.00              100.00                0.03                8040
PSU2   NOT OK                    0.00                  0.00                0.00                0.00                   0
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# show platform psustatus
PSU    Model      Serial          HW Rev      Voltage (V)    Current (A)    Power (W)  Status    LED
-----  ---------  --------------  --------  -------------  -------------  -----------  --------  -----
PSU 1  02RPHXA00  CNDED0003R07ZP  N/A               12.40           8.00       100.00  OK        green
PSU 2  02RPHXA00  CNDED0003R07YF  N/A                0.00           0.00         0.00  NOT OK    off
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# pddf_fanutil getspeed
FAN           SPEED (RPM)
----------  -------------
Fantray1_1           9120
Fantray1_2           8280
Fantray2_1           9000
Fantray2_2           8160
Fantray3_1           9120
Fantray3_2           8280
Fantray4_1           9120
Fantray4_2           8160
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# show platform fan
  Drawer    LED         FAN    Speed    Direction    Presence    Status          Timestamp
--------  -----  ----------  -------  -----------  ----------  --------  -----------------
Fantray1  green  Fantray1_1  9120RPM      Unknown     Present        OK  20230125 04:22:14
Fantray1  green  Fantray1_2  8160RPM      Unknown     Present        OK  20230125 04:22:15
Fantray2  green  Fantray2_1  9000RPM      Unknown     Present        OK  20230125 04:22:15
Fantray2  green  Fantray2_2  8160RPM      Unknown     Present        OK  20230125 04:22:33
Fantray3  green  Fantray3_1  9120RPM      Unknown     Present        OK  20230125 04:22:52
Fantray3  green  Fantray3_2  8280RPM      Unknown     Present        OK  20230125 04:23:12
Fantray4  green  Fantray4_1  9120RPM      Unknown     Present        OK  20230125 04:21:57
Fantray4  green  Fantray4_2  8160RPM      Unknown     Present        OK  20230125 04:21:57
     N/A    N/A   PSU1_FAN1      32%      Unknown     Present        OK  20230125 04:21:58
     N/A    N/A   PSU2_FAN1       0%      Unknown     Present    Not OK  20230125 04:21:58
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# pddf_thermalutil gettemp
Temp Sensor           Value
--------------------  -------
CPU On-board          temp1	 +30.0 C (high = +90.0 C, crit = +94.0 C)
ASIC On-board         temp1	 +33.0 C (high = +78.0 C, crit = +80.0 C)
System Front Left     temp1	 +26.0 C (high = +59.0 C, crit = +62.0 C)
System Front Middle   temp1	 +29.0 C ()
System Front Right    temp1	 +22.0 C ()
Inlet Airflow Sensor  temp1	 +25.0 C ()
PSU1 Airflow Sensor   temp1	 +25.0 C ()
PSU2 Airflow Sensor   temp1	 +27.0 C ()
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# show platform temperature
              Sensor    Temperature    High TH    Low TH    Crit High TH    Crit Low TH    Warning          Timestamp
--------------------  -------------  ---------  --------  --------------  -------------  ---------  -----------------
       ASIC On-board         33           78.0       N/A            80.0            N/A      False  20230125 04:21:58
        CPU On-board         30           90.0       N/A            94.0            N/A      False  20230125 04:21:58
Inlet Airflow Sensor         25            N/A       N/A             N/A            N/A      False  20230125 04:21:58
 PSU1 Airflow Sensor         25            N/A       N/A             N/A            N/A      False  20230125 04:21:58
          PSU1_TEMP1          0.031        N/A       N/A             N/A            N/A      False  20230125 04:21:58
 PSU2 Airflow Sensor         27            N/A       N/A             N/A            N/A      False  20230125 04:21:58
          PSU2_TEMP1          0            N/A       N/A             N/A            N/A      False  20230125 04:21:58
   System Front Left         26           59.0       N/A            62.0            N/A      False  20230125 04:21:58
 System Front Middle         29            N/A       N/A             N/A            N/A      False  20230125 04:21:58
  System Front Right         22            N/A       N/A             N/A            N/A      False  20230125 04:21:58
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# show platform syseeprom
TlvInfo Header:
   Id String:    TlvInfo
   Version:      1
   Total Length: 180
TLV Name             Code      Len  Value
-------------------  ------  -----  ------------------------------
Product Name         0x21        9  S5232F-ON
Part Number          0x22        6  01WJVT
Serial Number        0x23       20  CN01WJVTCES0006A0050
Base MAC Address     0x24        6  0C:29:EF:D7:DB:20
Manufacture Date     0x25       19  06/13/2020 09:44:45
Device Version       0x26        1  1
Label Revision       0x27        3  A03
Platform Name        0x28       30  X86_64-dellemc_s5232f_c3538-r0
ONIE Version         0x29       10  3.40.1.1-6
MAC Addresses        0x2A        2  384
Manufacturer         0x2B        5  CES00
Manufacture Country  0x2C        2  CN
Vendor Name          0x2D        8  Dell EMC
Diag Version         0x2E       10  3.40.4.1-6
Service Tag          0x2F        7  G0SC9Z2
Vendor Extension     0xFD        4  0x00 0x00 0x02 0xA2
CRC-32               0xFE        4  0x4F157A6F

(checksum valid)
root@sonic:/home/admin#
root@sonic:/home/admin#
root@sonic:/home/admin# sfputil show presence
Port         Presence
-----------  -----------
Ethernet0    Present
Ethernet4    Present
Ethernet8    Not present
Ethernet12   Not present
Ethernet16   Not present
Ethernet20   Not present
Ethernet24   Not present
Ethernet28   Not present
Ethernet32   Present
Ethernet36   Not present
Ethernet40   Not present
Ethernet44   Not present
Ethernet48   Not present
Ethernet52   Not present
Ethernet56   Present
Ethernet60   Present
Ethernet64   Not present
Ethernet68   Not present
Ethernet72   Not present
Ethernet76   Not present
Ethernet80   Not present
Ethernet84   Not present
Ethernet88   Not present
Ethernet92   Not present
Ethernet96   Not present
Ethernet100  Not present
Ethernet104  Not present
Ethernet108  Not present
Ethernet112  Not present
Ethernet116  Not present
Ethernet120  Not present
Ethernet124  Not present
Ethernet128  Not present
Ethernet129  Not present
root@sonic:/home/admin# sfputil show eeprom -p Ethernet0
Ethernet0: SFP EEPROM detected
        Application Advertisement: N/A
        Connector: MPO 1x12
        Encoding: NRZ
        Extended Identifier: Power Class 4 Module (3.5W max.), No CLEI code present in Page 02h, CDR present in TX, CDR present in RX
        Extended RateSelect Compliance: Rate Select Version 2
        Identifier: QSFP28 or later
        Length Cable Assembly(m): 50.0
        Nominal Bit Rate(100Mbs): 255
        Specification compliance:
                10/40G Ethernet Compliance Code: Extended
                Extended Specification Compliance: 100GBASE-SR4 or 25GBASE-SR
                Fibre Channel Link Length: Unknown
                Fibre Channel Speed: Unknown
                Fibre Channel Transmission Media: Unknown
                Fibre Channel Transmitter Technology: Unknown
                Gigabit Ethernet Compliant Codes: Unknown
                SAS/SATA Compliance Codes: Unknown
                SONET Compliance Codes: Unknown
        Vendor Date Code(YYYY-MM-DD Lot): 2019-10-17
        Vendor Name: FS
        Vendor OUI: 00-00-00
        Vendor PN: QSFP28-SR4-100G
        Vendor Rev: 1A
        Vendor SN: C1910106515


root@sonic:/home/admin#
root@sonic:/home/admin# sfputil show eeprom -p Ethernet4
Ethernet4: SFP EEPROM detected
        Application Advertisement: N/A
        Connector: MPO 1x12
        Encoding: NRZ
        Extended Identifier: Power Class 4 Module (3.5W max.), No CLEI code present in Page 02h, CDR present in TX, CDR present in RX
        Extended RateSelect Compliance: Rate Select Version 2
        Identifier: QSFP28 or later
        Length Cable Assembly(m): 50.0
        Nominal Bit Rate(100Mbs): 255
        Specification compliance:
                10/40G Ethernet Compliance Code: Extended
                Extended Specification Compliance: 100GBASE-SR4 or 25GBASE-SR
                Fibre Channel Link Length: Unknown
                Fibre Channel Speed: Unknown
                Fibre Channel Transmission Media: Unknown
                Fibre Channel Transmitter Technology: Unknown
                Gigabit Ethernet Compliant Codes: Unknown
                SAS/SATA Compliance Codes: Unknown
                SONET Compliance Codes: Unknown
        Vendor Date Code(YYYY-MM-DD Lot): 2020-06-06
        Vendor Name: FS
        Vendor OUI: 00-00-00
        Vendor PN: QSFP28-SR4-100G
        Vendor Rev: 1A
        Vendor SN: C2006059407


root@sonic:/home/admin#
root@sonic:/home/admin# show int st
  Interface            Lanes    Speed    MTU    FEC            Alias    Vlan    Oper    Admin                                      Type    Asym PFC
-----------  ---------------  -------  -----  -----  ---------------  ------  ------  -------  ----------------------------------------  ----------
  Ethernet0          1,2,3,4     100G   9100    N/A   hundredGigE1/1  routed    down       up                           QSFP28 or later         N/A
  Ethernet4          5,6,7,8     100G   9100    N/A   hundredGigE1/2  routed      up       up                           QSFP28 or later         N/A
  Ethernet8       9,10,11,12     100G   9100    N/A   hundredGigE1/3  routed    down       up                                       N/A         N/A
 Ethernet12      13,14,15,16     100G   9100    N/A   hundredGigE1/4  routed    down       up                                       N/A         N/A
..
..
root@sonic:/home/admin#

* NOTE *: FAN airflow direction and some of the other attributes are absent in above logs as those entries were not properly added in pddf-device.json file. This was just a test run to show that PDDF FPGA drivers work.

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111
  • 202205
  • 202211

Description for the changelog

Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

@adyeung
Copy link
Collaborator

adyeung commented Jan 24, 2023

@jostar-yang pls help review

@FuzailBrcm
Copy link
Contributor Author

/azpw

@FuzailBrcm
Copy link
Contributor Author

/azpw run

@mssonicbld
Copy link
Collaborator

/AzurePipelines run

@azure-pipelines
Copy link

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

@FuzailBrcm
Copy link
Contributor Author

/azp run Azure.sonic-buildimage

@azure-pipelines
Copy link

Commenter does not have sufficient privileges for PR 13477 in repo sonic-net/sonic-buildimage

@FuzailBrcm
Copy link
Contributor Author

/azp run Azure.sonic-buildimage

@azure-pipelines
Copy link

Commenter does not have sufficient privileges for PR 13477 in repo sonic-net/sonic-buildimage

@FuzailBrcm
Copy link
Contributor Author

/azpw run Azure.sonic-buildimage

@mssonicbld
Copy link
Collaborator

/AzurePipelines run Azure.sonic-buildimage

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@FuzailBrcm
Copy link
Contributor Author

FuzailBrcm commented Jan 31, 2023

The sonic-buildimage build failure is due to the dependency on other PRs viz. #13475 and #13476 . Once they gets merged, the build issue would be resolved.

I had to split the changes into multiple PRs for ease of review but the inter-dependency could not be avoided.

@FuzailBrcm
Copy link
Contributor Author

@celestica-gl , @qnos

Please review.

Copy link
Contributor

@jostar-yang jostar-yang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code is fine to me.

Copy link
Contributor

@qnos qnos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compile failure due to depend on PR#13475 and PR#13476, it's not a issue.

@FuzailBrcm FuzailBrcm force-pushed the pddf_fpgaenable_1 branch 2 times, most recently from e961ed2 to 980a71a Compare February 7, 2023 04:16
@FuzailBrcm
Copy link
Contributor Author

/azpw run

@mssonicbld
Copy link
Collaborator

/AzurePipelines run

@azure-pipelines
Copy link

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

@FuzailBrcm
Copy link
Contributor Author

/azp run Azure.sonic-buildimage

@azure-pipelines
Copy link

Commenter does not have sufficient privileges for PR 13477 in repo sonic-net/sonic-buildimage

@FuzailBrcm
Copy link
Contributor Author

/azpw run Azure.sonic-buildimage

@mssonicbld
Copy link
Collaborator

/AzurePipelines run Azure.sonic-buildimage

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@zhangyanzhao
Copy link
Collaborator

/azpw run Azure.sonic-buildimage

@FuzailBrcm
Copy link
Contributor Author

@pettershao-ragilenetworks
This build failure is in the debian compilation of Ragile platform.
The PDDF common code has changed little bit. Especially the LED common code. You will have to modify your code a little. Please check this sonic-net/SONiC#1232. It has all the code PRs.
If you are using custom LED driver based on the PDDF common drivers, you will have to adjust that code.

For now, I will reference the old header files in your platform path so that the compilation goes fine.

@FuzailBrcm
Copy link
Contributor Author

/azpw run Azure.sonic-buildimage

@mssonicbld
Copy link
Collaborator

/AzurePipelines run Azure.sonic-buildimage

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@FuzailBrcm
Copy link
Contributor Author

@pettershao-ragilenetworks
Please review the change

@adyeung
Copy link
Collaborator

adyeung commented Feb 28, 2023

@lguohan pls help merge the code PR

@prgeor prgeor added the PDDF label Mar 14, 2023
@yxieca yxieca merged commit f822373 into sonic-net:master Mar 15, 2023
qnos pushed a commit to qnos/sonic-buildimage that referenced this pull request Nov 12, 2023
Why I did it
To enable FPGA support in PDDF.

How I did it
Added FPGAI2C and FPGAPCI in the build path for the PDDF debian package
Added the support for FPGA access APIs in the drivers of fan, xcvr, led etc.
Added the FPGA device creation support in PDDF utils and parsers

How to verify it
These changes can be verified on some platform using such FPGAs. For testing purpose, we took Dell S5232f platform and brought it up using PDDF. In doing so, FPGA devices are created using PDDF and optics eeproms were accessed using common FPGA drivers. Below are some of the logs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants