Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Import patients zip file API via POST- /admin/patients is not working #180

Open
shardulbanker opened this issue Sep 25, 2019 · 21 comments
Open

Import patients zip file API via POST- /admin/patients is not working #180

shardulbanker opened this issue Sep 25, 2019 · 21 comments

Comments

@shardulbanker
Copy link

Hi,

I am testing POST- /admin/patients API using postman. I am passing below parameters as form data. If

  1. file - patient's zip file containing CDA XML file
  2. practice_id - Provider's ID
  3. practice_name - Provider Name

It is throwing below error.

Started POST "/api/admin/patients/" for 10.10.0.80 at 2019-09-25 08:57:28 -0400
Processing by Api::Admin::PatientsController#create as /
Can't verify CSRF token authenticity
Completed 500 Internal Server Error in 453ms

NoMethodError (undefined method read' for nil:NilClass): app/controllers/api/admin/patients_controller.rb:47:in block in create'
app/controllers/api/admin/patients_controller.rb:47:in open' app/controllers/api/admin/patients_controller.rb:47:in create'

However, when I try same zip file from popHealth UI it is working as expected and imported the patient. It increments the patients count and verify the same in MongoDB database in 'records' collection also.

Please look into it and suggest way forward as API is not working.

Thanks and regards,
Shardul
@petercyli
Copy link
Collaborator

petercyli commented Sep 25, 2019 via email

@shardulbanker
Copy link
Author

Hi Shardul, Please check out this documentation - https://github.com/OSEHRA/popHealth/wiki/Getting-Started-with-popHealth-API Make sure the enable_csrf_for_apis: false in the popHealth.yml file. Peter From: shardulbanker [email protected] Sent: Wednesday, September 25, 2019 9:10 AM To: OSEHRA/popHealth [email protected] Cc: Subscribed [email protected] Subject: [OSEHRA/popHealth] Import patients zip file API via POST- /admin/patients is not working (#180) Hi, I am testing POST- /admin/patients API using postman. I am passing below parameters as form data. If 1. file - patient's zip file containing CDA XML file 2. practice_id - Provider's ID 3. practice_name - Provider Name It is throwing below error. Started POST "/api/admin/patients/" for 10.10.0.80 at 2019-09-25 08:57:28 -0400 Processing by Api::Admin::PatientsController#create as / Can't verify CSRF token authenticity Completed 500 Internal Server Error in 453ms NoMethodError (undefined method read' for nil:NilClass): app/controllers/api/admin/patients_controller.rb:47:in block in create' app/controllers/api/admin/patients_controller.rb:47:in open' app/controllers/api/admin/patients_controller.rb:47:in create' However, when I try same zip file from popHealth UI it is working as expected and imported the patient. It increments the patients count and verify the same in MongoDB database in 'records' collection also. Please look into it and suggest way forward as API is not working. Thanks and regards, Shardul — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#180?email_source=notifications&email_token=AAMCG3L4OKNRYFDYORXUZ5LQLNPLJA5CNFSM4I2MTBCKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HNTDGTA> , or mute the thread https://github.com/notifications/unsubscribe-auth/AAMCG3JKV4W2P7MV6R4F6ZDQLNPLJANCNFSM4I2MTBCA .

Thanks for the update. This already know and went through that guide also earlier.

This error is just for logging purpose, it doesn't cause an issue. Actual issue is about below error...
NoMethodError (undefined method read' for nil:NilClass): app/controllers/api/admin/patients_controller.rb:47
This error throws at line no. which is highlighted in bold color in sample code snippet.

 api :POST, "/admin/patients", "Upload a zip file of patients."
  param :file, nil, :desc => 'The zip file of patients to upload.', :required => true
  param :practice_id, String, :desc => "ID for the patient's Practice", :required => false
  param :practice_name, String, :desc => "Name for the patient's Practice", :required => false

  def create
    log_admin_api_call LogAction::ADD, "Upload patient ZIP file", true
    **file = params[:file]**

Do we have any fix available for this issue? Can you please help me currently with any work around so that I can continue with importing patient via API? I am currently stuck and not able to import patient from our EHR application.

Thanks and regards,
Shardul

@petercyli
Copy link
Collaborator

petercyli commented Sep 25, 2019 via email

@petercyli
Copy link
Collaborator

I just used postman to import a patient zip file,
postman-import_patient
and it worked fine

@shardulbanker
Copy link
Author

@petercyli,

Thanks for update.

I am not using v6 version. I am using 6.0.0rc1 version. Please see version info collected from popHealth home page.

last tag: v6.0.0.rc1
bundle: 2017.0.4

I am using postman with below parameters.

file - patient's zip file containing CDA XML file
practice_id - Provider's ID
practice_name - Provider Name
Capture

It is giving 500 Internal server error and based on checking logs it gives below error highlighted in BOLD.

Started POST "/api/admin/patients/" for 10.10.0.80 at 2019-09-26 03:36:18 -0400
Processing by Api::Admin::PatientsController#create as /
Can't verify CSRF token authenticity
Completed 500 Internal Server Error in 418ms

NoMethodError (undefined method read' for nil:NilClass): app/controllers/api/admin/patients_controller_before_change.rb:47:in block in create'
app/controllers/api/admin/patients_controller_before_change.rb:47:in open' app/controllers/api/admin/patients_controller_before_change.rb:47:in create'

@shardulbanker
Copy link
Author

@petercyli ,
Below is the code snippet where it is throwing error in app/controllers/api/admin/patients_controller.rb at line no. 17. Here it gets file object as Nil and that's we can't apply read method on file.read on line no. 17 and gives this error.

I am wondering how we can get uploaded file object correctly and apply file.read operation on it.

Please suggest way to proceed further. I am not able to proceed further. Thanks.

  1. _api :POST, "/admin/patients", "Upload a zip file of patients."
  2.   param :file, nil, :desc => 'The zip file of patients to upload.', :required => true

  3.   param :practice_id, String, :desc => "ID for the patient's Practice", :required => false

  4.   param :practice_name, String, :desc => "Name for the patient's Practice", :required => false

  6.   def create

  7.     log_admin_api_call LogAction::ADD, "Upload patient ZIP file", true

  8.     file = params[:file]

  9.     practice = get_practice_parameter(params[:practice_id], params[:practice_name])

  11.     FileUtils.mkdir_p(File.join(Dir.pwd, "tmp/import"))

  12.     file_location = File.join(Dir.pwd, "tmp/import")

  13.     file_name = "patient_upload" + Time.now.to_i.to_s + rand(1000).to_s

  15.     temp_file = File.new(file_location + "/" + file_name, "w")

  17.     **_File.open(temp_file.path, "wb") { |f| f.write(file.read) }**

@petercyli
Copy link
Collaborator

Can you try the API without the practice id? I just used the practice name.

@shardulbanker
Copy link
Author

Tried as suggested through only practice name and file but same error. No improvement.

@shardulbanker
Copy link
Author

@petercyli ,

I have little bit modified the code of app/controller/api/admin/patients_controller.rb file and changed line no.8 as per code mentioned in earlier comment and changed to line.
Changed from
file = params[:file]
to
file = StringIO.new(request.body.read)

Now, it is not giving 500 internal server error. Now it fails later stage import patient job with below error.

/home/shardulb/osehra_pophealth/tmp/import/failed# vi patient_upload1569505510299.error
File /home/shardulb/osehra_pophealth/tmp/import/patient_upload1569505510299 has zero size. Did you mean to pass the create flag?
/usr/local/rvm/gems/ruby-2.2.5/gems/rubyzip-1.2.1/lib/zip/file.rb:82:in initialize' /usr/local/rvm/gems/ruby-2.2.5/gems/rubyzip-1.2.1/lib/zip/file.rb:98:in new'
/usr/local/rvm/gems/ruby-2.2.5/gems/rubyzip-1.2.1/lib/zip/file.rb:98:in open' /home/shardulb/osehra_pophealth/lib/hds/bulk_record_importer.rb:17:in import_archive'
/home/shardulb/osehra_pophealth/lib/import_archive_job.rb:16:in `perform'

There problem is related to that we are not passing file correctly to import job.

Can you please help how to pass correct file object to import job?

Another thing this error collected from failed directory. But internally it tries to search from /home/shardulb/osehra_pophealth/tmp/import/patient_upload1569505510299.

This file found in /home/shardulb/osehra_pophealth/tmp/import/failed.

Please help and let me know the correct method of passing file object to import job.

Thanks.

@shardulbanker
Copy link
Author

@petercyli

Could you please help?

@petercyli
Copy link
Collaborator

So the api method tried to import the zip file, but the popHealth had an error processing the zip file; hence, the error seen in the tmp subdirectory.
Were you able to import the same zip file using the popHealth UI via the admininstrator menu?

@shardulbanker
Copy link
Author

Yes. Same zip file is uploaded via popHealth UI and patient count is also increased. Only the problem with API.
In your case it is working as expected as per postman snapshot shared in comment.

Can you please share your code of app/controllers/api/admin/patients_controller.rb file?

@petercyli
Copy link
Collaborator

This is code from the latest v6 branch - https://github.com/OSEHRA/popHealth/blob/v6/app/controllers/api/admin/patients_controller.rb

@shardulbanker
Copy link
Author

Checked and verified latest v6 branch code and 6.0.0rc1 version and they are same. Nothing is changed. Even I put your file but same issue.

I am wondering how it works in your case.

I have question about how to read file body parameter correctly to get zip file so that it copies at location /tmp/import directory for further processing.

@shardulbanker
Copy link
Author

I have resolved all the errors. Now records is not inserted into records collection.
Which file it uses to insert the patient into record collection? Is there any way to enable debug/info logs and what's the syntax to add new logs in application?

@petercyli
Copy link
Collaborator

BTW, popHealth version 6 is not using the records collection. Patient data is stored in the qdm_patients collection. There is no extra configuration to generate more debug logs. We will need to add more debug statements to improve logging.

@shardulbanker
Copy link
Author

Currently when file is selected for processing using ImportArchiveJob it will remove it from tmp/import directory. Is there way I can uncomment the code of file removal so that I get the file at tmp/import directory. I need to check file and verify few things.

Could you please help code where it removes file from /tmp/import directory?

Thanks.

@shardulbanker
Copy link
Author

Please give me you email and I need to contact for further help. We can setup meeting for further discussion. I will show you all details with all options I have tried still API is not working.

I checked using UI and it was successful but API is not working.

@petercyli
Copy link
Collaborator

Can you send me the zip file containing the test files?

@shardulbanker
Copy link
Author

@petercyli
Thanks Peter for all your help. I have resolved the issue. Now I am able to upload patient's zip file using API. This API finally adds patient's record in MongoDB.

Can I contribute code in master branch? What are the steps to contribute the code?

@petercyli
Copy link
Collaborator

Definitely. You can create a pull request to the v6 branch in the popHealth. I will merge the code after reviewing the change.

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@petercyli @shardulbanker