⚠️ WORK IN PROGRESS⚠️
This repository contains a complete test environment for the Exchange System (ES, Italian: SDI) for Electronic Invoices, including implementations for the ES itself and for the other participants.
The test environment can be used to:
-
simulate the complete process of invoice issue, transmission and receipt, including all notifications and handling of anomalous situations
-
during the development of SDICoop compliant services, simulate, debug and test locally their interaction with the ES and other actors
-
develop SDICoop-compliant services, forking the IR (Issuer/Recipient) implementation
-
develop higher-level applications that interact with a SDICoop compliant service, i.e. user interfaces, invoice/notification archiving ...
At this stage the testsdi is WIP and not fully implemented, most importantly these features are missing:
Some functionalities are also excluded from the initial design:
- receiving / transmitting ZIP archives (see issue #25)
- receiving / transmitting invoices with multiple
FatturaElettronicaBody
elements ("multi-invoices") (see issue #22)
Index:
- Introduction
- Architecture
- Implementation
- Getting Started
- Testing
- Troubleshooting
- Contributing
- Authors
- License
The ES system run by the Italian government (SDICoop Service - Transmit / Receive) is used to transmit and receive invoices, receipts and notifications.
There are three actors:
-
issuer (Italian: trasmittente), implements 1 SOAP Web Service (WS):
- SDICoop Transmit / TrasmissioneFatture: to receive receipts and notifications from the ES
and uses the ES SDICoop Transmit / SdIRiceviFile endpoint to issue invoices
-
ES (forwards invoice from issuer to recipient Italian: SDI), implements 2 SOAP WS:
- SDICoop Transmit / SdIRiceviFile, gets invoices from the issuer
- SDICoop Receive / SdIRiceviNotifica: gets receipts from the recipient and forwards it to the issuer
and uses:
- and the issuer SDICoop Transmit / TrasmissioneFatture endpoint to send receipts notifications
- the recipent SDICoop Receive / RicezioneFatture endpoint to send invoices
-
and recipient (Italian: destinatario), implements 1 SOAP WS:
- SDICoop Receive / RicezioneFatture: to get invoices and notifications from the SDI
and uses the ES SdIRiceviNotifica WS to send notifications
The following animation shows the minimal workflow from invoice issue to receipt of acceptance:
This can also be seen grouping separately the two services:
SDICoop Transmit
SDICoop Receive
There is some English documentation but it's outdated. The Italian documentation is more up-to-date.
The testsdi is monolithic but modular, so that specific functionalities can be easily extracted.
A distinctive design choice has been to use the same database schema, API and structure for all actors. Rather than breaking it down in components based on the actors, it has been broken down in layers:
-
The SOAP server component exposes the interfaces required to communicate in accordance with the FatturaPA specification; it uses the fatturapa-core classes
-
The core component (fatturapa-core), has:
- state machine abstraction
- state persistency to database for each invoice and notification
Base
,Issuer
,Exchange
andRecipient
classes.- accesses the SOAP endpoints acting as SOAP client.
-
The control component (fatturapa-control), also uses fatturapa-core, and exposes a Remote Procedure Calls (RPC) API over the HTTP protocol. This API can be used to control the simulation / tests or to show status information in user interfaces.
-
The ui component fatturapa-ui provides a basic User Interface to interact with the test environment.
This picture shows how the 4 layers stack up:
This screencast demonstrates the complete workflow (see Demo section below) as seen through the UI, i.e. how you can send an invoice from I/R 0000001 to I/R 0000002, and make sure that the various notifications are sent back and forth between the three involved actors until the invoice acceptance is confirmed for all the parties:
The invoices change state during the workflow; certain state changes trigger notifications that have to be sent to specific actors.
The states are represented with strings starting with E_
for exchanger states, I_
for issuer states, R_
for recipient states and N_
for notification states.
The possible states, state changes and triggers are shown in the following state diagrams.
Legend for all state diagrams:
Status | Description |
---|---|
I_UPLOADED | ready to be transmitted |
I_TRANSMITTED | transmitted to ES |
I_DELIVERED | ES notified that it was delivered to Recipient |
I_FAILED_DELIVERY | failed delivery within 48 hours |
I_INVALID | ES notified it was invalid |
I_IMPOSSIBLE_DELIVERY | ES notified that it was not delivered to the recipient within 48 hours + 10 days |
I_ACCEPTED | ES notified that it was accepted by the recipient |
I_REFUSED | ES notified that it was refused by the recipient |
I_EXPIRED | ES notified that it was not accepted / refused by the recipient for more than 15 days |
Status | Description |
---|---|
E_RECEIVED = I_TRANSMITTED | received from transmitter |
E_VALID | passed checks |
E_FAILED_DELIVERY | failed delivery within 48 hours |
E_DELIVERED | delivered to recipient |
E_INVALID | did not pass test |
E_IMPOSSIBLE_DELIVERY | failed delivery within 48 hours + 10 days |
E_ACCEPTED | Recipient notified that it accepted the invoice |
E_REFUSED | Recipient notified that it refused the invoice |
E_EXPIRED | not accepted / refused by the recipient for more than 15 days |
Status | Description |
---|---|
R_RECEIVED = E_DELIVERED | received from ES |
R_ACCEPTED | Accepted |
R_REFUSED | Refused |
R_EXPIRED | ES notified that it was not accepted / refused by the recipient for more than 15 days |
Status | Description |
---|---|
N_RECEIVED | inbound notification has been received |
N_PENDING | outbound notification has been generated and must be processed |
N_OBSOLETE | outbound notification has been generated but must not be processed because another notification has been generated that makes notifcation of this one useless |
N_DELIVERED | outbound notification has beensuccessfully delivered |
There is a common database for all actors, consisting in three tables:
invoices:
- id: integer, primary key
- remote_id: the id of this invoice for the sdi actor
- invoice reference based on file and position:
- nomefile
- posizione
- invoice reference based on uniqueness of the mandatory invoice fields:
- cedente
- anno
- numero
- status: state machine status
- blob: base64-encoded blob of the invoice
- ctime: record creation time in database
- actor: the actor on behalf on whom we are storing the invoice, one of: sdi, td + 7-characters code
- issuer: 7-characters code of the original issuer of the invoice
notifications:
- id: integer, primary key
- invoice_id
- type: one of AttestazioneTrasmissioneFattura, NotificaDecorrenzaTermini, RicevutaConsegna, ...
- status: one of N_RECEIVED, N_PENDING, N_OBSOLETE, N_DELIVERED
- blob: base64-encoded blob of the notification
- actor: the actor on behalf on whom we are storing the notification, one of: sdi, td + 7-characters code
- nomefile
- ctime: record creation time in database
channels (lookup table between cedente
and issuer
):
- cedente: primary key
- issuer: 7-characters code for issuer which transmits invoices on behalf of cedente
For each of the four SOAP Web Services, we started from the Web Services Description Language, (WSDL) and XML Schema Definition, (XSD) files from fatturapa.gov.it, fed them to wsdl2phpgenerator which generated types and boilerplate for the endpoint in a directory named as the endpoint.
This code generation step has been performed once and for all by the soap/bin/generate.php script.
In each of the four resulting directory matching the endpoints, we place a index.php
file similar to (this one is for the SdIRiceviFile
endpoint):
require_once("SdIRiceviFileHandler.php");
$srv = new \SoapServer('SdIRiceviFile_v1.0.wsdl');
$srv->setClass("SdIRiceviFileHandler");
$srv->handle();
which leverages the PHP SoapServer class and delegates the implementation to a handler class SdIRiceviFileHandler
.
The handler class is implemented in a file with the same name SdIRiceviFileHandler.php
in the endpoint directory, and uses robust type cheching thanks to type hinting and the type declarations obtained from wsdl2phpgenerator.
Tested on: amd64 Debian 9.5 (stretch, current stable) with PHP 7.0 and Laravel 5.5.44.
Install prerequisites:
sudo apt install php-cli php-fpm nginx php-soap php-mbstring php-dom php-zip composer nginx postgresql php-pgsql php-curl php-xml
Clone the repo into the /var/www/html
directory on your webserver.
cd /var/www/html
git clone https://github.com/italia/fatturapa-testsdi .
Install prerequisites with composer:
cd /var/www/html
composer install
Configure the database:
- in
/etc/postgresql/9.6/main/pg_hba.conf
add this line:
local testsdi www-data md5
before this one:
# "local" is for Unix domain socket connections only
local all all peer
-
restart postgresql with:
sudo systemctl restart postgresql
-
Create the database:
sudo su - postgres
psql
CREATE USER "www-data" WITH PASSWORD 'www-data';
CREATE DATABASE testsdi OWNER "www-data";
^d
^d
You'll be able to access the database with:
PGPASSWORD="www-data" psql -U www-data testsdi
Configure database credentials in core/config.php
and in rpc/config/database.php
.
Configure HOSTNAME
in soap/config.php
and in core/config.php
.
Set up Laravel:
cd /var/www/html
sudo chown www-data core/storage/time_travel.json
cd rpc
touch storage/logs/laravel.log
sudo chown -R www-data storage/logs
sudo chmod g+w storage/logs/laravel.log
sudo chown -R www-data storage/framework
sudo chown -R www-data bootstrap/cache
cp .env.example .env
php artisan key:generate
^d
Configure nginx:
sudo rm /etc/nginx/sites-enabled/*
sudo vi /etc/nginx/sites-enabled/fatturapa
Set the contents of the /etc/nginx/sites-enabled/fatturapa
file to something like:
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name testsdi.simevo.com;
root /var/www/html;
index index.html index.htm index.php;
location ~ \.php$ {
include snippets/fastcgi-php.conf;
fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
fastcgi_read_timeout 300;
}
location ^~ /sdi/rpc/js/ {
alias /var/www/html/rpc/packages/fatturapa/ui/src/public/js/;
}
location ^~ /sdi/rpc/css/ {
alias /var/www/html/rpc/packages/fatturapa/ui/src/public/css/;
}
location ^~ /sdi/rpc/webfonts/ {
alias /var/www/html/rpc/packages/fatturapa/ui/src/public/webfonts/;
}
location ^~ /sdi/rpc/font/ {
alias /var/www/html/rpc/packages/fatturapa/ui/src/public/font/;
}
location ~ /.*/rpc {
try_files $uri $uri/ /rpc/index.php?$query_string;
}
location ~ /.*/soap {
try_files $uri $uri/ /soap/index.php;
}
}
Finally check the configuration and restart nginx:
sudo nginx -t
sudo systemctl restart nginx
At this point you should be able to access the UI at: https://testsdi.example.com/sdi/rpc/dashboard
Dynamic routing makes sure that the RPC endpoints for the actors will be reachable at:
/sdi
- the Exchange System (there's only one)/tdxxxxxxx
,/tdyyyyyyy
, ... - where td stands for trasmittente/destinatario (T/D), Italian for issuer/receiver (I/R) andxxxxxxx
,yyyyyyy
are the 7-characters I/R identification codes.
The number of simulated I/R (T/D) actors are autoconfigured based on the actors that appear in the channels
table.
For example if you set the channels table like this so that invoices can be sent (needed for the tests):
INSERT INTO channels(cedente, issuer) VALUES ('IT-01234567890', '0000001');
INSERT INTO channels(cedente, issuer) VALUES ('IT-12345678901', '0000002');
INSERT INTO channels(cedente, issuer) VALUES ('IT-23456789012', '0000003');
your SOAP endpoints will be at:
- exchange
- issuer / recipient 0000001:
- issuer / recipient 0000002:
- issuer / recipient 0000003:
Sample manual session to demonstrate the flow of one invoice from issuer 0000001 to recipient 0000002, and acceptance:
- clear status
POST https://www.example.com/sdi/rpc/clear
POST https://www.example.com/td0000001/rpc/clear
POST https://www.example.com/td0000002/rpc/clear
- create a valid sample invoice for TD 0000002 (
FatturaElettronica.FatturaElettronicaHeader.DatiTrasmissione.CodiceDestinatario
should be set to0000002
) and upload it to TD 0000001, then check it is in the right queue
POST https://www.example.com/td0000001/rpc/upload {file XML}
GET https://www.example.com/td0000001/rpc/invoices?status=I_UPLOADED
- force transmission to ES and check status:
POST https://www.example.com/td0000001/rpc/transmit
- Check status with ES (the invoice should be in the E_RECEIVED queue):
GET https://www.example.com/sdi/rpc/invoices?status=E_RECEIVED
- Check status with TD 0000001 (the invoice should be in the I_TRANSMITTED queue):
GET https://www.example.com/td0000001/rpc/invoices?status=I_TRANSMITTED
- force validation by ES and check status:
POST https://www.example.com/sdi/rpc/checkValidity
GET https://www.example.com/sdi/rpc/invoices?status=E_VALID
- force transmission from ES to recipient and check status:
POST https://www.example.com/sdi/rpc/deliver
GET https://www.example.com/sdi/rpc/invoices?status=E_DELIVERED
GET https://www.example.com/sdi/td0000002/invoices?status=R_RECEIVED
GET https://www.example.com/td0000001/rpc/invoices?status=I_DELIVERED (no response yet because ES has not notified to issuer)
- force ES to dispatch back the notification to the issuer:
POST https://www.example.com/sdi/rpc/dispatch
- check notification and status, now for the issuer TD 0000001 the invoice should be in the I_DELIVERED queue:
GET https://www.example.com/td0000001/rpc/notifications/id
GET https://www.example.com/td0000001/rpc/invoices?status=I_DELIVERED
- make recipient accept invoice and check status:
POST https://www.example.com/td0000002/rpc/accept/id
GET https://www.example.com/td0000002/rpc/invoices?status=R_ACCEPTED
GET https://www.example.com/sdi/rpc/invoices?status=E_ACCEPTED (no response yet)
- force receiver to dispatch back the notification to the ES:
POST https://www.example.com/td0000002/rpc/dispatch
- check notification and status:
GET https://www.example.com/sdi/rpc/notifications/id
GET https://www.example.com/sdi/rpc/invoices?status=E_ACCEPTED
GET https://www.example.com/td0000002/rpc/invoices?status=I_ACCEPTED (no response yet)
- force ES to dispatch back the acceptance notification to the issuer:
POST https://www.example.com/sdi/rpc/dispatch
- check notification and status:
GET https://www.example.com/td0000001/rpc/notifications/id
GET https://www.example.com/td0000002/rpc/invoices?status=I_ACCEPTED
You can test manually by making SOAP requests using Postman.
You can import this collection into Postman, to test the AttestazioneTrasmissioneFattura
operation of the TrasmissioneFatture
web service (change the url to match that of your test server !):
{
"variables": [],
"info": {
"name": "SOAP",
"_postman_id": "0ee991f3-5203-a8ac-6b38-32c8bfabc05e",
"description": "",
"schema": "https://schema.getpostman.com/json/collection/v2.0.0/collection.json"
},
"item": [
{
"name": "SDICoop Transmit / TrasmissioneFatture service, AttestazioneTrasmissioneFattura operation",
"request": {
"url": "http://testsdi.simevo.com/td0000001/soap/TrasmissioneFatture/",
"method": "POST",
"header": [
{
"key": "Content-Type",
"value": "text/xml;charset=\"utf-8\"",
"description": ""
},
{
"key": "SOAPAction",
"value": "http://www.fatturapa.it/TrasmissioneFatture/AttestazioneTrasmissioneFattura",
"description": ""
}
],
"body": {
"mode": "raw",
"raw": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<SOAP-ENV:Envelope\n xmlns:SOAP-ENV=\"http://schemas.xmlsoap.org/soap/envelope/\"\n xmlns:ns1=\"http://www.fatturapa.gov.it/sdi/ws/trasmissione/v1.0/types\">\n\t<SOAP-ENV:Body>\n\t\t<ns1:attestazioneTrasmissioneFattura>\n\t\t\t<IdentificativoSdI>104</IdentificativoSdI>\n\t\t\t<NomeFile>IT01234567890_11111_AT_001.xml</NomeFile>\n\t\t\t<File>UEQ5NGJXd2dkbVZ5YzJsdmJqMGlNUzR3SWlCbGJtTnZaR2x1WnowaVZWUkdMVGdpUHo0S1BEOTRiV3d0YzNSNWJHVnphR1ZsZENCMGVYQmxQU0owWlhoMEwzaHpiQ0lnYUhKbFpqMGlRVlJmZGpFdU1TNTRjMndpUHo0S1BIUjVjR1Z6T2tGMGRHVnpkR0Y2YVc5dVpWUnlZWE50YVhOemFXOXVaVVpoZEhSMWNtRWdlRzFzYm5NNmRIbHdaWE05SW1oMGRIQTZMeTkzZDNjdVptRjBkSFZ5WVhCaExtZHZkaTVwZEM5elpHa3ZiV1Z6YzJGbloya3ZkakV1TUNJZ2VHMXNibk02ZUhOcFBTSm9kSFJ3T2k4dmQzZDNMbmN6TG05eVp5OHlNREF4TDFoTlRGTmphR1Z0WVMxcGJuTjBZVzVqWlNJZ2RtVnljMmx2Ym1VOUlqRXVNQ0lnZUhOcE9uTmphR1Z0WVV4dlkyRjBhVzl1UFNKb2RIUndPaTh2ZDNkM0xtWmhkSFIxY21Gd1lTNW5iM1l1YVhRdmMyUnBMMjFsYzNOaFoyZHBMM1l4TGpBZ1RXVnpjMkZuWjJsVWVYQmxjMTkyTVM0eExuaHpaQ0FpUGdvOFNXUmxiblJwWm1sallYUnBkbTlUWkVrK01qRTBQQzlKWkdWdWRHbG1hV05oZEdsMmIxTmtTVDRLUEU1dmJXVkdhV3hsUGtsVU1ERXlNelExTmpjNE9UQmZNVEV4TVRFdWVHMXNMbkEzYlR3dlRtOXRaVVpwYkdVK0NqeEVZWFJoVDNKaFVtbGpaWHBwYjI1bFBqSXdNVFF0TURRdE1ERlVNVEk2TURBNk1EQThMMFJoZEdGUGNtRlNhV05sZW1sdmJtVStDanhFWlhOMGFXNWhkR0Z5YVc4K0NpQWdJQ0E4UTI5a2FXTmxQa0ZCUVVGQlFUd3ZRMjlrYVdObFBnb2dJQ0FnUEVSbGMyTnlhWHBwYjI1bFBsQjFZbUpzYVdOaElFRnRiV2x1YVhOMGNtRjZhVzl1WlNCa2FTQndjbTkyWVR3dlJHVnpZM0pwZW1sdmJtVStDand2UkdWemRHbHVZWFJoY21sdlBnbzhUV1Z6YzJGblpVbGtQakV5TXpRMU5qd3ZUV1Z6YzJGblpVbGtQZ284VG05MFpUNUJkSFJsYzNSaGVtbHZibVVnVkhKaGMyMXBjM05wYjI1bElFWmhkSFIxY21FZ1pHa2djSEp2ZG1FOEwwNXZkR1UrQ2p4SVlYTm9SbWxzWlU5eWFXZHBibUZzWlQ0eVl6Rm1NMkV5TkRCaE1EVTJaRGsxTXpkaE9EWXdPR1psWkRNeE1EZ3hNbVZtTjJJeFlqZGhOREV3WkRBeE5USm1OV001WXpsbE9UTTBPRFpoWlRRMFBDOUlZWE5vUm1sc1pVOXlhV2RwYm1Gc1pUNEtQQzkwZVhCbGN6cEJkSFJsYzNSaGVtbHZibVZVY21GemJXbHpjMmx2Ym1WR1lYUjBkWEpoUGc9PQ==</File>\n\t\t</ns1:attestazioneTrasmissioneFattura>\n\t</SOAP-ENV:Body>\n</SOAP-ENV:Envelope>"
},
"description": ""
},
"response": []
}
]
}
Test the PHP classes with:
phpunit --testdox tests
This project complies with the PSR-2: Coding Style Guide.
Lint the code with:
./vendor/bin/phpcs --standard=PSR2 xxx.php
SOAP client/server interactions can be tricky to debug.
The issue is even more complicated when you perform a RPC calls (such as POST /sdi/rpc/transmit
) which has to perform internally a SOAP call.
You'll then have: client -> 1st level server (RPC) | SOAP client -> SOAP server | 2nd level server.
It is easy to trace and debug the 1st level server:
- PHP
echo
is sent back to client - PHP
error_log
statements get written to/var/log/nginx/error.log
.
For the 2nd level server it's more complicated:
- you cannot use PHP
echo
because the body is used to return XML payloads - PHP
error_log
statements get lost.
To make sure you get to see all messages written to log at the 2nd level server (SOAP server) edit /etc/php/7.0/fpm/pool.d/www.conf
adding these lines at the end:
catch_workers_output = yes
php_flag[display_errors] = on
php_admin_value[error_log] = /var/log/fpm-php.www.log
php_admin_flag[log_errors] = on
then create the new log file and make sure it can be written by the webserver:
sudo touch /var/log/fpm-php.www.log
sudo chown www-data:www-data /var/log/fpm-php.www.log
and finally restart the servers:
systemctl restart nginx && systemctl restart php7.0-fpm
Another option you have is to use instrumented versions of the PHP SoapClient
and SoapServer
builtins.
To instrument a SOAP client, use SoapClientDebug
instead of SoapClient
, for example for TrasmissioneFatture
add this to soap/TrasmissioneFatture/autoload.php
:
'SoapClientDebug' => __DIR__ .'/../SoapClientDebug.php',
then modify TrasmissioneFatture/TrasmissioneFatture_service.php
like this:
- class TrasmissioneFatture_service extends \SoapClient
+ class TrasmissioneFatture_service extends \SoapClientDebug
To instrument a SOAP server, use SoapServerDebug
instead of SoapServer
, for example for TrasmissioneFatture
make sure soap/index.php
has:
require_once("SoapServerDebug.php");
then modify TrasmissioneFatture/index.php
like this:
-$srv = new \SoapServer(dirname(__FILE__) . '/TrasmissioneFatture_v1.1.wsdl');
+$srv = new SoapServerDebug(dirname(__FILE__) . '/TrasmissioneFatture_v1.1.wsdl');
$srv->setClass("TrasmissioneFattureHandler");
$srv->handle();
+foreach ($srv->getAllDebugValues() as $value) {
+ error_log('==== '. print_r($value, true));
+}
For your contributions please use the git-flow workflow.
Emanuele Aina, Riccardo Mariani, Marco Peca and Paolo Greppi.
Copyright (c) 2018, simevo s.r.l.
License: AGPL 3, see LICENSE file.