keyboard() and visually type() does not work with non-English - new solution [done]

[alexandrelsantana]

r.type () does not work when the text uses a special character (ç, á, etc.). Example: r.type ('text_desc.png', 'descrição')
How can I resolve this error?

Thaks

[alexandrelsantana]

adicional information in moment de error:

[PravinAlhat]

Hi @alexandrelsantana
I am able to type a string descrição into a text field. Using Pycham IDE for scripting so not sure about jupyter notebook.
The below code works absolutely fine with Python3.8.1 on Mac.

import rpa as r
r.init()
r.url('https://www.google.com')
r.type('q','descrição[enter]')

Just give a try with visual automation OFF/False?

[alexandrelsantana]

Hi @PravinAlhat
I started using Pycharm IDE with python version 3.9.1 and the error persists. If I change to "False", the VISUAL AUTOMATION automation type does not work.
I did a test using "WEB AUTOMATION" with the same code and it worked. The error must be when using VISUAL AUTOMATION with special characters, because when I remove "ç" and "ã" it works perfectly.

Any other suggestions?

Thank you for your help!

[PravinAlhat]

Hi @alexandrelsantana
I tried with the below code in Pycharm,

import rpa as r
r.init(visual_automation=True)
r.url('https://www.google.com')
r.wait(2)
print(r.exist('Input.png'))
print(r.exist('Search.png'))
r.type('Input.png','descrição')
r.click('Search.png')
r.close()

Observation:
print(r.exist('Input.png')). --- True
print(r.exist('Search.png')) --- True

Even though It found the images on the screen, it didn't type anything in the input text field rather my execution was halted as shown in the below image. I could only see a blinking cursor in my Pycharm terminal. However, It is clicking on the search button with visual automation (Here I have commented the line r.type('Input.png','descrição') just expected my script to click on the Search button).
I am using a Macbook pro so due to its retina display, sometimes Sikuli doesn't identify/evaluate the captured images.

Thank you for your inputs and do let me know if you find any solution for this.

[kensoh]

Hi RPA for Python users, 🅗🅐🅟🅟🅨 ②⓪②①!

It has been many months since I last worked on this personal side project. Just want to shout out that this project is very much alive, and I hope to look into the 30+ issues raised since July (and bug-fixes if required) soon when I get a moment again.

Below are details if you are interested, why I'm away past few months and why I believe my darkest days are over 🍀🤞🏻🙏🏻

---

Since May last year, I've stopped all work to juggle with my mum's critical illness, taking care of my toddler, and doing housework chores like cleaning and cooking. TBH, my wife and I felt that taking care of a baby and giving her the best nutrition (nursing and home-cooked organic food) and care requires at least 1.5 full-time headcount. There is no life to speak of, other than trying to stay afloat with baby care needs, and keep finding alternative ways to treat my mum, where there're some wins.

Eg importing soy-based tube feeds from Netherlands / Germany / Australia --> after switching away from milk-based tube feeds for 2 weeks she stops requiring dialysis. She started requiring 3x a week dialysis for more than 3 months, after admitting to the hospital. Also, applying herbs like mashed garlic or diced onions to her feet somehow consistently improve her breathing and lung function. Have to keep trying alternative healing methods when her recovery keeps hitting plateau in hospital.

She has been discharged from the hospital a month ago and recovering at home, though very slowly. A longterm medication seems to be one of the contributing causes for her condition. After stopping it, there seems to be improvement to her breathing capacity. She is still bed-bound, but over the course of the year I hope she'll regain strength to move about, and hopefully lungs to recover sufficiently to not require oxygen tank support through a tracheal mask (a hole through her throat).

---

Since November, I've gone back to work at AI Singapore, a government-funded programme to accelerate AI in Singapore. I'm with Product Engineering team, focusing on RPA and TagUI (open-source RPA tool), which RPA for Python is a wrapper for. There are pretty interesting stuffs on TagUI 2021 roadmap. 2020 was the year which my family was going to move to the UK and work, after I receive its tier 1 exceptional tech talent visa. Then everything changed with the virus outbreak, and my mum's illness. The best thing that happened to me in 2020 was being able to go back to work at AI Singapore - I'm grateful.

I've just hired someone to help with housework, cooking and supporting my wife with baby care. I hope this new arrangement will allow me to deliver on my work at AI Singapore, and soon be able to reply countless friends' messages which circumstances leave me with no choice but go missing for months, and then look into the issues raised here!

Eg of the cost of best-in-class nutrition standard researched and set by wife. Baby just felt restless and broke her plate. Even Corelle's break-resistant plate breaks. Why not use plastic plate? Cuz plastic can leech toxins to the food on it. Why not use Silicon plate? Cuz there is no white silicon materials for plate. Why white? Cuz that helps to improve visibility of the food instead of contrasting against some pink colour or some other colours. I would be spending much time clearing up the mess of 70 over pieces of plate fragments, while baby gets nursed to calm down, if not for our newfound helper.

That's why I feel like my darkest days are over and now seeing glimpses of light at the end of the tunnel.

---

My vision for RPA for Python remains - I would like it to be the go-to Python package for RPA.

A best-in-class tool in Python users toolbox which you combine with other cool packages to build amazing stuffs.

[kensoh]

Hi @alexandrelsantana and @PravinAlhat, this is a limitation with SikuliX, the engine use for low-level keyboard and mouse control. SikuliX is also used for the visual automation capability through OpenCV and Tesseract. However, SikuliX is coded to work on a US-layout keyboard, thus simulating characters on non-English keyboard will not work.

There is however a workaround. Above commit in v1.32 enables copying non-English characters to clipboard. After you pip install rpa --upgrade, can you try if the following works for you? It works by putting non-English text onto the clipboard, and the pastes in using keyboard shortcut key. This way, SikuliX doesn't need to simulate non-English characters.

# -*- coding: utf-8 -*-
import rpa as r
r.init(True)
# click into some field, eg r.click('search.png')
r.clipboard('descrição')
r.keyboard('[ctrl]v')
r.close()

[alexandrelsantana]

Hi, @kensoh
I installed version 1.32 and it didn't work.

thaks!

Version:

Code:

import rpa as r

r.init(visual_automation=True)
r.type('buscar.png', 'descrição')
r.close()

Error:

[kensoh]

Hi @alexandrelsantana the example I give is a workaround, by copying the text you want into clipboard then pasting it. The type() function cannot visually type using non-English characters due to limitation in SikuliX engine. Below workaround bypasses this limitation by pasting text into somewhere you want instead of typing.

Below is sample script in my previous reply -

# -*- coding: utf-8 -*-
import rpa as r
r.init(True)
# click into some field, eg r.click('search.png')
r.clipboard('descrição')
r.keyboard('[ctrl]v')
r.close()

[alexandrelsantana]

Hi @kensoh! Thaks, but not work.

import rpa as r

r.init(True)
r.click('buscar.png')
r.clipboard('descrição')
r.keyboard('[ctrl]v')

r.close()

[kensoh]

Hi @alexandrelsantana did you include the first line in the example I give above?

For some Python version, to write non-English characters, need this line # -*- coding: utf-8 -*-