Skip to content

Commit

Permalink
added faker help
Browse files Browse the repository at this point in the history
  • Loading branch information
jruaux committed Jun 8, 2019
1 parent 0d03035 commit 689cdff
Show file tree
Hide file tree
Showing 36 changed files with 188 additions and 659 deletions.
76 changes: 71 additions & 5 deletions README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ riot import json --url https://raw.githubusercontent.com/rethinkdb/beerthink/mas
==== People
[source,shell]
----
riot import --max 100 gen --field id=sequence --field firstName=faker.name.firstName --field lastName=faker.name.lastName --field address=faker.address.fullAddress hash --keyspace person --keys id
riot import faker --max 100 --field id=sequence --field firstName=name.firstName --field lastName=name.lastName --field address=address.fullAddress hash --keyspace person --keys id
----

[source,plaintext]
Expand All @@ -204,7 +204,7 @@ riot import --max 100 gen --field id=sequence --field firstName=faker.name.first
==== Game of Thrones
[source,shell]
----
riot import --max 100 gen --field name=faker.gameOfThrones.character set --keyspace got:characters --fields name
riot import faker --max 100 --field name=gameOfThrones.character set --keyspace got:characters --fields name
----

[source,plaintext]
Expand All @@ -220,9 +220,75 @@ riot import --max 100 gen --field name=faker.gameOfThrones.character set --keysp

==== Faker Generators

These are the https://github.com/DiUS/java-faker[Faker generators] available:

`address, ancient, app, artist, avatar, beer, book, bool, business, cat, chuckNorris, code, color, commerce, company, country, crypto, currency, dateAndTime, demographic, dog, dragonBall, educator, esports, file, finance, food, friends, funnyName, gameOfThrones, hacker, harryPotter, hipster, hitchhikersGuideToTheGalaxy, hobbit, howIMetYourMother, idNumber, internet, job, leagueOfLegends, lebowski, lordOfTheRings, lorem, matz, medical, music, name, number, options, overwatch, phoneNumber, pokemon, randomService, rickAndMorty, robin, rockBand, shakespeare, slackEmoji, space, starTrek, stock, superhero, team, twinPeaks, university, weather, witcher, yoda, zelda`
https://github.com/DiUS/java-faker[Faker] generators:

* *address*: state timeZone lastName country latitude longitude firstName streetAddressNumber streetName zipCode streetAddress secondaryAddress stateAbbr streetSuffix streetPrefix citySuffix cityPrefix city cityName countryCode buildingNumber fullAddress
* *ancient*: primordial titan god hero
* *app*: name version author
* *artist*: name
* *avatar*: image
* *beer*: name yeast malt hop style
* *book*: title publisher author genre
* *bool*: bool
* *business*: creditCardNumber creditCardType creditCardExpiry
* *cat*: name registry breed
* *chuckNorris*: fact
* *code*: asin isbnGs1 isbnGroup isbnRegistrant isbn10 isbn13 imei ean8 gtin8 ean13 gtin13
* *color*: name hex
* *commerce*: productName material price promotionCode department color
* *company*: name suffix url industry profession buzzword catchPhrase logo domainName bs domainSuffix
* *country*: flag currencyCode currency countryCode2 countryCode3 capital
* *crypto*: sha1 sha256 md5 sha512
* *currency*: name code
* *date*: birthday
* *demographic*: educationalAttainment demonym race sex maritalStatus
* *dog*: name size memePhrase age sound coatLength gender breed
* *dragonBall*: character
* *educator*: secondarySchool course campus university
* *esports*: event player league game team
* *file*: fileName extension mimeType
* *finance*: creditCard bic iban randomCreditCardType createCountryCodeToBasicBankAccountNumberPatternMap
* *food*: spice ingredient measurement
* *friends*: location character quote
* *funnyName*: name
* *gameOfThrones*: character dragon house quote city
* *hacker*: abbreviation adjective noun verb ingverb
* *harryPotter*: location character quote book
* *hipster*: word
* *hitchhikersGuideToTheGalaxy*: location character specie marvinQuote starship quote planet
* *hobbit*: location character thorinsCompany quote
* *howIMetYourMother*: character catchPhrase highFive quote
* *idNumber*: valid ssnValid validSvSeSsn invalidSvSeSsn invalid
* *internet*: url uuid domainName password image avatar slug emailAddress safeEmailAddress domainWord domainSuffix macAddress ipV4Address privateIpV4Address publicIpV4Address ipV4Cidr ipV6Address ipV6Cidr userAgentAny
* *job*: field position title keySkills seniority
* *leagueOfLegends*: location summonerSpell masteries champion rank quote
* *lebowski*: character actor quote
* *lordOfTheRings*: location character
* *lorem*: character words word characters sentence paragraph
* *matz*: quote
* *medical*: medicineName symptoms diseaseName hospitalName
* *music*: key instrument chord genre
* *name*: name prefix suffix title lastName fullName firstName nameWithMiddle username
* *number*: digit randomDigit randomDigitNotZero randomNumber
* *overwatch*: location quote hero
* *phoneNumber*: phoneNumber cellPhone
* *pokemon*: name location
* *rickAndMorty*: location character quote
* *robin*: quote
* *rockBand*: name
* *shakespeare*: hamletQuote asYouLikeItQuote kingRichardIIIQuote romeoAndJulietQuote
* *slackEmoji*: people nature activity foodAndDrink celebration travelAndPlaces objectsAndSymbols custom emoji
* *space*: company planet moon galaxy nebula starCluster constellation star agency agencyAbbreviation meteorite nasaSpaceCraft distanceMeasurement
* *starTrek*: location character specie villain
* *stock*: nyseSymbol nsdqSymbol
* *superhero*: name prefix suffix descriptor power
* *team*: name state sport creature
* *twinPeaks*: location character quote
* *university*: name prefix suffix
* *weather*: temperatureCelsius temperatureFahrenheit description
* *witcher*: location character school monster quote witcher
* *yoda*: quote
* *zelda*: character game

=== Export to Databases

Expand Down
32 changes: 0 additions & 32 deletions src/main/java/com/redislabs/riot/cli/CommandsBuilder.java

This file was deleted.

56 changes: 56 additions & 0 deletions src/main/java/com/redislabs/riot/cli/in/GeneratorHelp.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
package com.redislabs.riot.cli.in;

import java.lang.reflect.Method;
import java.lang.reflect.Parameter;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

import org.springframework.core.convert.Property;

import com.github.javafaker.Faker;

import picocli.CommandLine.Command;

@Command(name = "faker-help", description = "Print all faker entities")
public class GeneratorHelp implements Runnable {

private final static List<String> EXCLUDES = Arrays.asList("instance", "options");

@Override
public void run() {
Arrays.asList(Faker.class.getDeclaredMethods()).stream().filter(this::accept)
.sorted((m1, m2) -> m1.getName().compareTo(m2.getName())).forEach(this::print);
}

public boolean accept(Method method) {
if (EXCLUDES.contains(method.getName())) {
return false;
}
return method.getReturnType().getPackage().equals(Faker.class.getPackage());
}

public void print(Method method) {
System.out.println(method.getName() + ": "
+ String.join(" ",
Arrays.asList(method.getReturnType().getDeclaredMethods()).stream()
.filter(m -> m.getParameters().length == 0).map(m -> describe(m))
.collect(Collectors.toList())));
}

private String describe(Method method) {
String description = method.getName();
if (method.getParameters().length > 0) {
description += "(";
description += String.join(", ",
Arrays.asList(method.getParameters()).stream().map(p -> describe(p)).collect(Collectors.toList()));
description += ")";
}
return description;
}

private String describe(Parameter parameter) {
return parameter.getType().getSimpleName();
}

}
17 changes: 11 additions & 6 deletions src/main/java/com/redislabs/riot/cli/in/GeneratorImport.java
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,15 @@
import java.util.Locale;
import java.util.Map;

import org.springframework.expression.Expression;
import org.springframework.expression.spel.standard.SpelExpressionParser;

import com.redislabs.riot.generator.GeneratorReader;
import com.redislabs.riot.generator.GeneratorReaderBuilder;

import picocli.CommandLine.Command;
import picocli.CommandLine.Option;

@Command(name = "gen", description = "Import randomly generated data")
@Command(name = "faker", description = "Import randomly generated data")
public class GeneratorImport extends AbstractImportReaderCommand {

@Option(names = "--map", description = "SpEL expression to generate maps.", paramLabel = "<SpEL>")
Expand All @@ -23,10 +25,13 @@ public class GeneratorImport extends AbstractImportReaderCommand {

@Override
public GeneratorReader reader() {
GeneratorReaderBuilder builder = new GeneratorReaderBuilder();
builder.setFields(fieldExpressions);
builder.setLocale(locale);
return builder.build();
SpelExpressionParser parser = new SpelExpressionParser();
Map<String, Expression> fieldExpressionMap = new LinkedHashMap<String, Expression>();
fieldExpressions.forEach((k, v) -> fieldExpressionMap.put(k, parser.parseExpression(v)));
GeneratorReader reader = new GeneratorReader();
reader.setFieldExpressions(fieldExpressionMap);
reader.setLocale(locale);
return reader;
}

@Override
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/com/redislabs/riot/cli/in/Import.java
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

@Command(name = "import", description = "Import into Redis", subcommands = { DelimitedFileImport.class,
FixedLengthFileImport.class, JsonFileImport.class, DatabaseImport.class, GeneratorImport.class,
SimpleGeneratorImport.class })
GeneratorHelp.class, SimpleGeneratorImport.class })
public class Import extends JobCommand<Map<String, Object>, Map<String, Object>> {

}
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

import picocli.CommandLine.Command;

@Command(name = "sgen", description = "Import simple generated data")
@Command(name = "simple", description = "Import simple generated data")
public class SimpleGeneratorImport extends AbstractImportReaderCommand {

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

import com.redislabs.riot.cli.in.AbstractImportWriterCommand;
import com.redislabs.riot.redis.writer.LettuceWriter;
import com.redislabs.riot.redis.writer.search.AbstractRediSearchItemWriter;
import com.redislabs.riot.redis.writer.search.AbstractLettuSearchItemWriter;

import lombok.Getter;
import picocli.CommandLine.Option;
Expand Down Expand Up @@ -35,13 +35,13 @@ protected ItemWriter<Map<String, Object>> writer() {

protected abstract ItemWriter<Map<String, Object>> jedisSearchWriter();

private AbstractRediSearchItemWriter lettuceItemWriter() {
AbstractRediSearchItemWriter writer = rediSearchItemWriter();
private AbstractLettuSearchItemWriter lettuceItemWriter() {
AbstractLettuSearchItemWriter writer = rediSearchItemWriter();
writer.setIndex(index);
writer.setConverter(redisConverter());
return writer;
}

protected abstract AbstractRediSearchItemWriter rediSearchItemWriter();
protected abstract AbstractLettuSearchItemWriter rediSearchItemWriter();

}
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import com.redislabs.lettusearch.search.AddOptions;
import com.redislabs.lettusearch.search.Language;
import com.redislabs.riot.redis.writer.search.JedisSearchWriter;
import com.redislabs.riot.redis.writer.search.SearchAddWriter;
import com.redislabs.riot.redis.writer.search.LettuSearchAddWriter;

import io.redisearch.client.AddOptions.ReplacementPolicy;
import io.redisearch.client.Client;
Expand Down Expand Up @@ -36,8 +36,8 @@ public class SearchImport extends AbstractRediSearchImport {
private String payload;

@Override
protected SearchAddWriter rediSearchItemWriter() {
SearchAddWriter writer = new SearchAddWriter();
protected LettuSearchAddWriter rediSearchItemWriter() {
LettuSearchAddWriter writer = new LettuSearchAddWriter();
writer.setDefaultScore(defaultScore);
writer.setOptions(AddOptions.builder().ifCondition(ifCondition).language(language).noSave(noSave)
.replace(replace).replacePartial(partial).build());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@

import org.springframework.batch.item.ItemWriter;

import com.redislabs.riot.redis.writer.search.AbstractRediSearchItemWriter;
import com.redislabs.riot.redis.writer.search.SuggestWriter;
import com.redislabs.riot.redis.writer.search.AbstractLettuSearchItemWriter;
import com.redislabs.riot.redis.writer.search.LettuSearchSuggestWriter;

import picocli.CommandLine.Command;
import picocli.CommandLine.Option;
Expand All @@ -25,8 +25,8 @@ public class SuggestImport extends AbstractRediSearchImport {
private String payload;

@Override
protected AbstractRediSearchItemWriter rediSearchItemWriter() {
SuggestWriter writer = new SuggestWriter();
protected AbstractLettuSearchItemWriter rediSearchItemWriter() {
LettuSearchSuggestWriter writer = new LettuSearchSuggestWriter();
writer.setDefaultScore(defaultScore);
writer.setField(suggest);
writer.setIncrement(increment);
Expand Down
28 changes: 28 additions & 0 deletions src/main/java/com/redislabs/riot/generator/GeneratorFaker.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
package com.redislabs.riot.generator;

import java.util.Locale;

import com.github.javafaker.Faker;

public class GeneratorFaker extends Faker {

private GeneratorReader reader;

public GeneratorFaker(Locale locale, GeneratorReader reader) {
super(locale);
this.reader = reader;
}

public long sequence() {
return reader.getSequence();
}

public int partitions() {
return reader.getPartitions();
}

public int partitionIndex() {
return reader.getPartitionIndex();
}

}
Loading

0 comments on commit 689cdff

Please sign in to comment.