Support for Nodejs Jasmine Tests

[mmraghavendra]

We are working on UI automation using Protractor ( https://www.protractortest.org ). We use Jasmine ( https://jasmine.github.io/setup/nodejs.html ) as our test runner.

We could make use of Meissa for test execution if you could add support. I met you in Selenium Conference 2018 and discussed the same with you.

Below is example of simple protractor test using Jasmine.
https://github.com/angular/protractor/tree/master/example

Please let me know if you need more info.

Thanks,
Raghav

[angelovstanton]

Hi Raghav,

Thank you for writing. It was a pleasure to meet you.

To develop the plug-in, I will need a little more help since I am not a JS developer. I have like basic knowledge of JS.

So, can you give me detailed instructions on what I need to install on my machine and how to run the examples you supplied? Can you also tell me more about your use case- how many tests do you have and what your setup is.
How can we send a list of tests to the Jasmin runner? Guess we can start it from the console?

In Meissa, we need a way to get all tests from your files? Maybe we can parse them or is there a better way? In C# and Java, we use reflection to extract all test methods that we later supply to native test runners.

Anton

[mmraghavendra]

You need to install latest nodejs from https://nodejs.org.
Create a directory say test and place the files in https://github.com/angular/protractor/tree/master/example

Then open command prompt in test directory and run below command.
npm install protractor
node_modules/.bin/webdriver-manager update

Now to run tests,
node_modules/.bin/protractor conf.js

Jasmine identifies the tests by spec file names. In https://github.com/angular/protractor/blob/master/example/conf.js : line 15, we provide the file names as a list.

All the it blocks (lines between 2-10) in https://github.com/angular/protractor/blob/master/example/example_spec.js are treated as tests. In jasmine, we can not run the tests within a spec file in parallel. Multiple spec files can be run in parallel.

In Meissa, you can treat each spec file as a test. If you clone example_spec.js as example_spec.1js, example_spec2.js etc and show me how to run these spec files in parallel, it would be helpful.

Thanks,
Raghav

[mmraghavendra]

Do you need any clarifications?

[angelovstanton]

Thank you for the detailed info. Sorry for not writing immediately, was traveling for a two weeks and didn't have internet most of the time. I will find time during the weekend to work on that and will give you an update if I don't understand something.

Have a nice day!

[mmraghavendra]

Thank you..

[angelovstanton]

Okay, I managed to setup and execute the tests. Read a bit Protractor and Jasmine docs but have a few questions.
I couldn't find how to generate a tests results file. Since we need it to merge the results, find failures etc. It will be perfect if you can tell me how to use - https://www.npmjs.com/package/jasmine-reporters
The NUnitXmlReporter will be perfect since we already have logic for merging NUnit tests results and we will reuse it.
As far as I understood, describe and it- specify the different tests which can be placed in more than one file. So my question here is, how we can filter the specs, I saw that there should be --filter="a spec name" arg but couldn't make it work. All 3 tests are executed.
The goal here is I guess if you have multiple files, we will parse them and will get the names of all specs (string in describe and it functions) and then we will filter them in multiple runs. When we have multiple results file we will parse and merge them. Is this logic correct or I don't understand fully how this works?

Anton

[mmraghavendra]

To filter spec using protractor, you need to use --grep option. But it will be treated as a regular expression and all matching specs will be executed. grep will be consider name of describe and it.

The goal you mentioned is correct.

I have attached code files updated to generate NUnit test results.
protractor-example.zip

Notice the changes in conf.js file, reading spec name from environment variable if provided.
specs: [process.env.SPECS ||'example_spec.js'],

When you collect multiple files, to run tests in a given file. Set the SPECS environment variable to file name and then run tests.

ex : set SPECS='example_spec2.js'
node_modules.bin\protractor conf.js // will run tests in example_spec2.js file

I am available for a skype call in detail if required.

Thanks,
Raghavendra

[angelovstanton]

Sorry, for not writing immediately, for some reason I didn't receive email notifications. Guess they have some bug here. Anyhow, I will check now your instructions and will write after.

Thanks,
Anton

[angelovstanton]

Okay, I managed to executed the tests and save them to NUnit.xml results file. But not quite sure how to use the filter option. Can you give an example how to execute 1 of these 3 specs? Is there a way to fully specify the names- like suite.specName so that some test not to be executed twice.

About ex command:
"D:\protractor-tests>ex : set SPECS='example_spec2.js'
'ex' is not recognized as an internal or external command,"

Maybe will be easier to talk on skype, mine is - withfullspeed

Thanks

[mmraghavendra]

Just execute below command..
set SPECS='example_spec2.js'

There is no way to specify suite.specName.. Same you can achieve by setting the SPECS like above to filter files and then using grep option to filter spec in the spec file.

Try this, we can have Skype if this does not work.

[angelovstanton]

Hey man,

Yesterday, I finished the plugin. Now released a beta version. You can try it from here.

You need to use the new testTechnology=Protractor

A few other things you need to know- do not manually install the NUnit Jasmin reporter, the plugin installed it in the temp conf.js files.

For library path- you need to point the path to your conf.js file.

As we talked, I didn't include the protractor folder in the plugins since you may want to install other dependencies. So, for everything to work, you need to make all installations in the Meissa\Plugins folder (node_modules) with the latest updated version of Protractor. The plugin gets the path to protractor as a relative path starting from the Plugins folder.

I wrote the merge logic for test result files as good as I could considering the simple example tests files. I tested it, and it should work. But, you are the man to tell me 100% if it is working as expected. Keep in mind that it is best if the tests names (it) are unique since in the tests report there are not IDs, I update them based on these names.

Please try it and tell me if everything is OK. When you confirm I will move the release to productions since now, I marked it as beta. Feel free to drop me a line in Skype if you prefer.

Best,
Anton

[angelovstanton]

@mmraghavendra Did you have the chance to test the new plugin? Or you are still travelling? :)

[mmraghavendra]

Sorry for the late reply. I have downloaded the latest beta zip file. I am able to start server and agent, but not getting how to execute tests. Can you please provide detailed steps?

[angelovstanton]

Okay, maybe I didn't explain well enough in the first message. Second try.

A few things need to be done before you can start the Meissa agent.

1. Open MeissaFolder/Plugins/ , make all installations for Protractor there (all steps you previously told me), no need to copy your conf.js or test files here, only install protractor.

2. DO NOT include NUnit Jasmin reporter module to your conf.js, the plugin does it automatically

3. Start Meissa in agent mode, as you normally do.

4. Start runner with similar arguments:

meissa.exe runner --resultsFilePath="D:\protractor-tests\mergedResult.xml" --outputFilesLocation="D:\protractor-tests" --agentTag="TestAgent" --testTechnology="Protractor" --testLibraryPath="D:\protractor-tests\conf.js" --testServerUrl="http://127.0.0.1:5000"

outputFilesLocations - all tests and other JS related files are here
testLibraryPath - path to your conf.js file where all tests are described
testTechnology - should be 'Protractor'

If you want distributed run, you have to start Meissa agent on two separate machines and repeat the operation with the installation of protractor. At the end of the run, you will have all test results merged into the file mergedResult.xml

[mmraghavendra]

I am seeing errors in Agent and agent is getting restarted. Please refer attached snapshots.

Please let me know how this works. I am seeing a temp directory getting created with contents of outputFilesLocation. Node modules directory in temp has only mkdirp and jasmine-reporters modules.

While outputFilesLocation i specified had node modules directory with many dependent modules.

[mmraghavendra]

attached dump file
08-20-2018-02-41-17-2490.txt

Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\examples\pow.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\chmod.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\clobber.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\mkdirp.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\perm.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\perm_sync.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\race.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\rel.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\return.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\return_sync.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\root.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\sync.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\umask.js
Copying C:\Users\testuser\AppData\Local\Temp\11d4e8e4-55b1-4d7a-aae6-81b14b
8f49d1\node_modules\mkdirp\test\umask_sync.js
The system cannot find the path specified.