Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] fix: replace missing Oralce XE image #187

Closed
wants to merge 10 commits into from
Closed

[WIP] fix: replace missing Oralce XE image #187

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
98e17ce
fix: replace missing Oralce XE image
Aug 30, 2019
b540b89
fix erroneous syntax issues
Sep 5, 2019
1589f4a
fix bash script
Sep 11, 2019
9888b65
UTC offset is messing up time
Sep 11, 2019
3553839
run setup on CI env
Sep 11, 2019
1118ce8
simplify setup.sh
Sep 11, 2019
27d141f
no build
Sep 13, 2019
b4b35a1
add build
Sep 15, 2019
e006d10
user creds
Sep 18, 2019
e8e7186
edit
Sep 18, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -464,12 +464,17 @@ npm test
```

### Docker

[ODPI-C](https://oracle.github.io/odpi/doc/installation.html) is a prerequisite, install on your system
before performing the next steps.

If you do not have a local Oracle instance, you can also run the test suite with very minimal requirements.
- Assuming you have [Docker](https://docs.docker.com/engine/installation/) installed, run the following script which would spawn an Oracle instance on your local machine:

```bash
source setup.sh <HOST> <PORT>
```
where `<HOST>`, `<PORT>`, `<USER>`, and `PASSWORD` are optional parameters. The default values are `localhost`, `1521`, `admin`, and `0raclep4ss` respectively. The `DATABASE` setting is always `XE`.
where `<HOST>`, `<PORT>`, `<USER>`, and `PASSWORD` are optional parameters. The default values are `localhost`, `1521`, `system`, and `oracle` respectively. The `DATABASE` setting is always `XE`.
- Run the test:
```bash
npm test
Expand Down
12 changes: 6 additions & 6 deletions lib/oracle.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,13 +349,13 @@ function dateToOracle(val, dateOnly) {
}

let dateStr = [
val.getUTCFullYear(),
fz(val.getUTCMonth() + 1),
fz(val.getUTCDate()),
val.getFullYear(),
fz(val.getMonth() + 1),
fz(val.getDate()),
].join('-') + ' ' + [
fz(val.getUTCHours()),
fz(val.getUTCMinutes()),
fz(val.getUTCSeconds()),
fz(val.getHours()),
fz(val.getMinutes()),
fz(val.getSeconds()),
].join(':');

if (!dateOnly) {
Expand Down
3 changes: 2 additions & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@
"scripts": {
"lint": "eslint .",
"lint:fix": "eslint . --fix",
"test": "mocha --UV_THREADPOOL_SIZE=100 test/*.test.js node_modules/juggler-v3/test.js node_modules/juggler-v4/test.js",
"pretest": "source ./setup.sh",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we need to have the setup.sh as the pretest, because it will imply that whoever running the tests need to use the oracle instance in docker.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we setup Oracle for CI?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rmg, do you know?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI is using a static test Oracle instance and injecting the credentials for the tests to pick them up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, using source here isn't portable. You should either run it directly (./setup.sh) or via bash (bash ./setup.sh).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rmg apart from an Oracle instance, we have to populate some data in it. I got setup.sh to do it's work (pulling the desired Oracle image, running it, and populating data), but ORACLE_USER and ORACLE_PASSWORD are not available to the app, despite export ORACLE_USER=SYSTEM ORACLE_PASSWORD=oracle && mocha --UV_THREADPOOL_SIZE=100 test/*.test.js node_modules/juggler-v3/test.js node_modules/juggler-v4/test.js.

On the "Environment variables" page of the project on Jenkins, there's no entry for ORACLE_USER and ORACLE_PASSWORD, while it is there for Cloudant, DashDB, Redis etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the static instance already has the data populated, we will not need to run setup.sh, just need ORACLE_USER and ORACLE_PASSWORD set in the "Environment variables" for the project.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The credentials are already provided via rc. The tests all have a line like this in them:

const config = require('rc')('loopback', {test: {oracle: {}}}).test.oracle;

That is pulling values from ~/.loopbackrc, which is being injected in the test environment specifically to provide those credentials. The instance should already contain the test data as far as I'm aware. The purpose of setup.sh was to create a local DB that matches that environment.

"test": "export ORACLE_USER=SYSTEM ORACLE_PASSWORD=oracle && mocha --UV_THREADPOOL_SIZE=100 test/*.test.js node_modules/juggler-v3/test.js node_modules/juggler-v4/test.js",
"posttest": "npm run lint"
},
"dependencies": {
Expand Down
45 changes: 9 additions & 36 deletions setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ CYAN='\033[1;36m'
PLAIN='\033[0m'

## variables
ORACLE_CONTAINER="oracle_c"
ORACLE_CONTAINER="oracle-xe"
HOST="localhost"
PORT=1521
DATABASE="XE"
USER="admin"
PASSWORD="0raclep4ss"
USER="SYSTEM"
PASSWORD="oracle"
if [ "$1" ]; then
HOST=$1
fi
Expand Down Expand Up @@ -46,12 +46,12 @@ printf "\n${CYAN}Clean up complete.${PLAIN}\n"

## pull latest oracle image
printf "\n${RED}>> Pulling latest oracle image${PLAIN} ${GREEN}...${PLAIN}"
docker pull sath89/oracle-xe-11g:latest > /dev/null 2>&1
docker pull webdizz/oracle-xe-11g-sa:latest > /dev/null 2>&1
printf "\n${CYAN}Image successfully built.${PLAIN}\n"

## run the oracle container
printf "\n${RED}>> Starting the oracle container${PLAIN} ${GREEN}...${PLAIN}\n"
docker run --name $ORACLE_CONTAINER -p $PORT:1521 -d sath89/oracle-xe-11g:latest > /dev/null 2>&1
docker run --name $ORACLE_CONTAINER -p $PORT:1521 -d webdizz/oracle-xe-11g-sa:latest > /dev/null 2>&1

##wait for orale database container to be ready
OUTPUT=1
Expand Down Expand Up @@ -86,40 +86,13 @@ printf "\n${RED}>> Exporting schema to database${PLAIN} ${GREEN}...${PLAIN}\n"
## copy over our db seed file
docker cp ./test/tables.sql $ORACLE_CONTAINER:/home/ > /dev/null 2>&1

##make user, give it privileges, and copy sql file to container
CREATEUSER="CREATE USER ${USER} IDENTIFIED by \"${PASSWORD}\";\n \
GRANT CONNECT, RESOURCE, DBA TO ${USER};\n \
GRANT CREATE SESSION TO ${USER};\n \
GRANT UNLIMITED TABLESPACE TO ${USER};\r"
## create the tables
printf "\n${GREEN}Waiting for database to respond with updated schema ..."
docker exec $ORACLE_CONTAINER /bin/sh -c "echo exit | sqlplus ${USER}/${PASSWORD}@//${HOST}:${PORT}/${DATABASE} @/home/tables.sql"

touch dockerusercreate.sql && echo ${CREATEUSER} > dockerusercreate.sql
docker cp dockerusercreate.sql $ORACLE_CONTAINER:/home/ > /dev/null 2>&1
rm dockerusercreate.sql
## run create user script
docker exec -it $ORACLE_CONTAINER /bin/sh -c "echo exit | sqlplus sys/oracle@//${HOST}:${PORT}/${DATABASE} as sysdba @/home/dockerusercreate.sql" > /dev/null 2>&1


## variables needed to health check export schema
OUTPUT=$?
TIMEOUT=120
TIME_PASSED=0
WAIT_STRING="."

printf "\n${GREEN}Waiting for database to respond with updated schema $WAIT_STRING${PLAIN}"
while [ "$OUTPUT" -ne 0 ] && [ "$TIMEOUT" -gt 0 ]
do
docker exec -it $ORACLE_CONTAINER /bin/sh -c "echo exit | sqlplus ${USER}/{$PASSWORD}@//${HOST}:${PORT}/${DATABASE} @/home/tables.sql" > /dev/null 2>&1
OUTPUT=$?
sleep 1s
TIMEOUT=$((TIMEOUT - 1))
TIME_PASSED=$((TIME_PASSED + 1))
if [ "$TIME_PASSED" -eq 5 ]; then
printf "${GREEN}.${PLAIN}"
TIME_PASSED=0
fi
done

if [ "$TIMEOUT" -le 0 ]; then
if [ "$OUTPUT" -ne 0 ]; then
printf "\n\n${CYAN}Status: ${PLAIN}${RED}Failed to export schema. Terminating setup.${PLAIN}\n\n"
exit 1
fi
Expand Down
2 changes: 2 additions & 0 deletions test/init/init.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ let DataSource = juggler.DataSource;

const config = require('rc')('loopback', {test: {oracle: {}}}).test.oracle;
config.maxConn = 64;
config.user = process.env.ORACLE_USER;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is throwing away the credentials provided by CI.

config.password = process.env.ORACLE_PASSWORD;

let db;

Expand Down
4 changes: 2 additions & 2 deletions test/oracle.autoupdate.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ describe('Oracle connector', function() {
options: {
idInjection: false,
oracle: {
schema: 'TEST',
schema: 'SYSTEM',
table: 'CUSTOMER_TEST',
},
},
Expand Down Expand Up @@ -55,7 +55,7 @@ describe('Oracle connector', function() {
options: {
idInjection: false,
oracle: {
schema: 'TEST',
schema: 'SYSTEM',
table: 'CUSTOMER_TEST',
},
},
Expand Down
2 changes: 1 addition & 1 deletion test/oracle.clob.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ before(function(done) {
name: 'ClobTest',
options: {
oracle: {
schema: 'TEST',
schema: 'SYSTEM',
table: 'CLOB_TEST',
},
},
Expand Down
2 changes: 2 additions & 0 deletions test/oracle.connectionpool.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ let db, config;

before(function() {
config = require('rc')('loopback', {dev: {oracle: {}}}).dev.oracle;
config.user = process.env.ORACLE_USER;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is throwing away the credentials provided by CI.

config.password = process.env.ORACLE_PASSWORD;
});

after(function() {
Expand Down
18 changes: 10 additions & 8 deletions test/oracle.discover.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ let db;
describe('discoverModels', function() {
before(function() {
const config = require('rc')('loopback', {dev: {oracle: {}}}).dev.oracle;
config.user = process.env.ORACLE_USER;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is throwing away the credentials provided by CI.

config.password = process.env.ORACLE_PASSWORD;
db = new DataSource(require('../'), config);
});

Expand Down Expand Up @@ -97,7 +99,7 @@ describe('discoverModels', function() {
let others = false;
models.forEach(function(m) {
// console.dir(m);
if (m.owner !== 'STRONGLOOP') {
if (m.owner !== 'SYSTEM') {
others = true;
}
});
Expand Down Expand Up @@ -126,7 +128,7 @@ describe('discoverModels', function() {
});

it('should return an array of columns for PRODUCT ', function(done) {
db.discoverModelProperties('PRODUCT', {schema: 'STRONGLOOP'},
db.discoverModelProperties('PRODUCT', {schema: 'SYSTEM'},
function(err, models) {
if (err) {
console.error(err);
Expand Down Expand Up @@ -159,9 +161,9 @@ describe('discoverModels', function() {
});
});

it('should return an array of primary keys for STRONGLOOP.PRODUCT',
it('should return an array of primary keys for SYSTEM.PRODUCT',
function(done) {
db.discoverPrimaryKeys('PRODUCT', {owner: 'STRONGLOOP'},
db.discoverPrimaryKeys('PRODUCT', {owner: 'SYSTEM'},
function(err, models) {
if (err) {
console.error(err);
Expand Down Expand Up @@ -192,9 +194,9 @@ describe('discoverModels', function() {
}
});
});
it('should return an array of foreign keys for STRONGLOOP.INVENTORY',
it('should return an array of foreign keys for SYSTEM.INVENTORY',
function(done) {
db.discoverForeignKeys('INVENTORY', {owner: 'STRONGLOOP'},
db.discoverForeignKeys('INVENTORY', {owner: 'SYSTEM'},
function(err, models) {
if (err) {
console.error(err);
Expand All @@ -212,11 +214,11 @@ describe('discoverModels', function() {

describe('Discover LDL schema from a table', function() {
it('should return an LDL schema for INVENTORY', function(done) {
db.discoverSchema('INVENTORY', {owner: 'STRONGLOOP'},
db.discoverSchema('INVENTORY', {owner: 'SYSTEM'},
function(err, schema) {
// console.log('%j', schema);
assert(schema.name === 'Inventory');
assert(schema.options.oracle.schema === 'STRONGLOOP');
assert(schema.options.oracle.schema === 'SYSTEM');
assert(schema.options.oracle.table === 'INVENTORY');
assert(schema.properties.productId);
assert(schema.properties.productId.type === 'String');
Expand Down
2 changes: 1 addition & 1 deletion test/oracle.mapping.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ describe('Mapping models', function() {
options: {
idInjection: false,
oracle: {
schema: 'TEST', 'table': 'INVENTORY_TEST',
schema: 'SYSTEM', 'table': 'INVENTORY_TEST',
},
},
properties: {
Expand Down
4 changes: 2 additions & 2 deletions test/oracle.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
'use strict';

/* global getDataSource */
var juggler = require('loopback-datasource-juggler');
var CreateDS = juggler.DataSource;
const juggler = require('loopback-datasource-juggler');
const CreateDS = juggler.DataSource;

require('./init/init');
const should = require('should');
Expand Down
38 changes: 19 additions & 19 deletions test/tables.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
-- File created - Thursday-September-05-2013
--------------------------------------------------------
DROP TABLE CUSTOMER CASCADE CONSTRAINTS PURGE;
DROP TABLE SESSION CASCADE CONSTRAINTS PURGE;
DROP TABLE SESS CASCADE CONSTRAINTS PURGE;
DROP TABLE LOCATION CASCADE CONSTRAINTS PURGE;
DROP TABLE PRODUCT CASCADE CONSTRAINTS PURGE;
DROP TABLE INVENTORY CASCADE CONSTRAINTS PURGE;
Expand All @@ -28,7 +28,7 @@ DROP TABLE RESERVATION CASCADE CONSTRAINTS PURGE;
"CREATED" DATE,
"LASTUPDATED" DATE
) ;
/

--------------------------------------------------------
-- DDL for Table INVENTORY
--------------------------------------------------------
Expand All @@ -40,7 +40,7 @@ DROP TABLE RESERVATION CASCADE CONSTRAINTS PURGE;
"AVAILABLE" NUMBER(22,0),
"TOTAL" NUMBER(22,0)
) ;
/


-- CREATE SEQUENCE INVENTORY_ID_SEQ START WITH 1 INCREMENT BY 1 NOMAXVALUE;
-- CREATE OR REPLACE TRIGGER INVENTORY_ID_TRIGGER BEFORE
Expand All @@ -60,7 +60,7 @@ DROP TABLE RESERVATION CASCADE CONSTRAINTS PURGE;
"NAME" VARCHAR2(32),
"GEO" VARCHAR2(1024)
) ;
/

--------------------------------------------------------
-- DDL for Table PRODUCT
--------------------------------------------------------
Expand All @@ -74,7 +74,7 @@ DROP TABLE RESERVATION CASCADE CONSTRAINTS PURGE;
"EXTRAS" VARCHAR2(64),
"FIRE_MODES" VARCHAR2(64)
) ;
/

--------------------------------------------------------
-- DDL for Table RESERVATION
--------------------------------------------------------
Expand All @@ -90,17 +90,17 @@ DROP TABLE RESERVATION CASCADE CONSTRAINTS PURGE;
"PICKUP_DATE" DATE,
"RETURN_DATE" DATE
) ;
/

--------------------------------------------------------
-- DDL for Table SESSION
-- DDL for Table SESS
--------------------------------------------------------

CREATE TABLE "SESSION"
CREATE TABLE "SESS"
( "ID" VARCHAR2(64),
"UID" VARCHAR2(1024),
"TTL" NUMBER
) ;
/

REM INSERTING into CUSTOMER
SET DEFINE OFF;
Insert into CUSTOMER (ID,USERNAME,EMAIL,PASSWORD,NAME,MILITARY_AGENCY,REALM,EMAILVERIFIED,VERIFICATIONTOKEN,CREDENTIALS,CHALLENGES,STATUS,CREATED,LASTUPDATED) values ('612','bat','[email protected]','$2a$10$BEG18wcYQn7TRkFIc59EB.vmnsEwqJWMlYM4DNG73iZb.MKA1rjAC',null,null,null,null,null,'[]','[]',null,null,null);
Expand Down Expand Up @@ -723,7 +723,7 @@ Insert into PRODUCT (ID,NAME,AUDIBLE_RANGE,EFFECTIVE_RANGE,ROUNDS,EXTRAS,FIRE_MO
Insert into PRODUCT (ID,NAME,AUDIBLE_RANGE,EFFECTIVE_RANGE,ROUNDS,EXTRAS,FIRE_MODES) values ('5','M9 SD',0,75,15,'Silenced','Single');
REM INSERTING into RESERVATION
SET DEFINE OFF;
REM INSERTING into "SESSION"
REM INSERTING into "SESS"
SET DEFINE OFF;

--------------------------------------------------------
Expand All @@ -746,31 +746,31 @@ CREATE OR REPLACE VIEW INVENTORY_VIEW
--------------------------------------------------------

ALTER TABLE "CUSTOMER" ADD PRIMARY KEY ("ID") ENABLE;
/

--------------------------------------------------------
-- Constraints for Table INVENTORY
--------------------------------------------------------

ALTER TABLE "INVENTORY" ADD PRIMARY KEY ("ID") ENABLE;
/

--------------------------------------------------------
-- Constraints for Table LOCATION
--------------------------------------------------------

ALTER TABLE "LOCATION" ADD PRIMARY KEY ("ID") ENABLE;
/

--------------------------------------------------------
-- Constraints for Table PRODUCT
--------------------------------------------------------

ALTER TABLE "PRODUCT" ADD PRIMARY KEY ("ID") ENABLE;
/

--------------------------------------------------------
-- Constraints for Table SESSION
-- Constraints for Table SESS
--------------------------------------------------------

ALTER TABLE "SESSION" ADD PRIMARY KEY ("ID") ENABLE;
/
ALTER TABLE "SESS" ADD PRIMARY KEY ("ID") ENABLE;

--------------------------------------------------------
-- Ref Constraints for Table INVENTORY
--------------------------------------------------------
Expand All @@ -779,7 +779,7 @@ CREATE OR REPLACE VIEW INVENTORY_VIEW
REFERENCES "LOCATION" ("ID") ENABLE;
ALTER TABLE "INVENTORY" ADD CONSTRAINT "PRODUCT_FK" FOREIGN KEY ("PRODUCT_ID")
REFERENCES "PRODUCT" ("ID") ENABLE;
/

--------------------------------------------------------
-- Ref Constraints for Table RESERVATION
--------------------------------------------------------
Expand All @@ -790,4 +790,4 @@ CREATE OR REPLACE VIEW INVENTORY_VIEW
REFERENCES "LOCATION" ("ID") ENABLE;
ALTER TABLE "RESERVATION" ADD CONSTRAINT "RESERVATION_PRODUCT_FK" FOREIGN KEY ("PRODUCT_ID")
REFERENCES "PRODUCT" ("ID") ENABLE;
/