Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate Scala Code #134

Open
arunmastermind opened this issue Sep 15, 2019 · 9 comments
Open

Generate Scala Code #134

arunmastermind opened this issue Sep 15, 2019 · 9 comments

Comments

@arunmastermind
Copy link

HI,

Hope you are doing great.
I would like to get your advise on if I could use your tool to generate scala codes.
Kindly suggest if that's possible.

Regards
AK

@ChrisCummins
Copy link
Owner

Hi Arun,

That's an interesting question. I've no experience with Scala myself, and I'm not aware of anyone who is working on it for CLgen, but I don't see any fundamental the approach that worked for OpenCL would not also work for Scala. You would need to assemble a training corpus of Scala code, then train a model on it and sample.

Cheers,
Chris

@arunmastermind
Copy link
Author

Hi Chris,

Thanks for coming back.
I have the training set, just need some advise on how does this model gets train.
do we have any instructions (layers, length of the code etc) for parameters.
I was sure that this would work for scala too but just wanted to be sure for training.
could you advise?

Regards
AK

@ChrisCummins
Copy link
Owner

Oh cool! Sounds like you're in a good position :) Unfortunately we haven't yet done a deep study on the effects of the model parameters on the quality of the generated samples. As a starting point I'd suggest simply taking the "tiny" model config supplied with this project and changing the LSTM size to a value that I have found that works for OpenCL: 1024 neurons per layer, 2 layers. Then keep an eye on the loss value during training and the quality of the model output every few epochs after the first 10. The documentation could be much better here. I'm currently working on a few extensions which should make tracking model training a little easier, e.g. by plotting training loss and providing regular model outputs during training etc. They aren't ready for release just yet though.

I'd be very interested to here how you get on!

Cheers,
Chris

@arunmastermind
Copy link
Author

Hi Chris,

I am getting below error while trying to run the tool:
/home/arunkumar/clgen# bazel run //deeplearning/clgen -- / --config $PWD/deeplearning/clgen/tests/data/tiny/config.pbtxt
Starting local Bazel server and connecting to it...
.................................
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:org_golang_x_tools: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:com_github_golang_protobuf: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:com_github_mwitkow_go_proto_validators: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:com_github_gogo_protobuf: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:org_golang_x_net: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:org_golang_x_text: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:org_golang_x_sys: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:org_golang_google_grpc: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:org_golang_google_genproto: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:go_googleapis: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:com_github_golang_glog: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: /root/.cache/bazel/_bazel_root/5d2e6cd8fc98586653a678599a58ba13/external/io_bazel_rules_go/go/private/repositories.bzl:237:9: //external:com_github_kevinburke_go_bindata: no such attribute 'patch_args' in 'repo_rule' rule
ERROR: error loading package '': Encountered error while reading extension file 'python/deps.bzl': no such package '@build_stack_rules_proto//python': error loading package 'external': Could not load //external package
ERROR: error loading package '': Encountered error while reading extension file 'python/deps.bzl': no such package '@build_stack_rules_proto//python': error loading package 'external': Could not load //external package
INFO: Elapsed time: 1.783s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)

Do you have any idea on the issue here.

Regards
AK

@ChrisCummins
Copy link
Owner

Hi Arun,

Hmm, looks like it might be a change in Bazel. What version of bazel are you using? For reference, I'm using 0.28.1. You might want to give that a try.

Cheers,
Chris

@arunmastermind
Copy link
Author

HiChris,

I have tried with 0.28.1 as well as 0.29.1 but no luck. I am still getting the same error.
I hope as per the error its expecting some directory as /external.
Could you please check and confirm.

Regards
Arun Kumar

@ChrisCummins
Copy link
Owner

The external/ directory is created by bazel when building external dependencies. It looks like the error you're hitting is caused by some extensions to support the golang programming language, but since that isn't required for this project you could try removing them in the WORKSPACE file? Like this:

diff --git a/WORKSPACE b/WORKSPACE
index b3727a490..8e48d985f 100644
--- a/WORKSPACE
+++ b/WORKSPACE
@@ -124,35 +124,6 @@ http_archive(
     urls = ["https://github.com/github/git-sizer/releases/download/v1.3.0/git-sizer-1.3.0-darwin-386.zip"],
 )
 
-# Golang and gazelle.
-
-load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
-
-http_archive(
-    name = "io_bazel_rules_go",
-    sha256 = "f04d2373bcaf8aa09bccb08a98a57e721306c8f6043a2a0ee610fd6853dcde3d",
-    urls = [
-        "https://storage.googleapis.com/bazel-mirror/github.com/bazelbuild/rules_go/releases/download/0.18.6/rules_go-0.18.6.tar.gz",
-        "https://github.com/bazelbuild/rules_go/releases/download/0.18.6/rules_go-0.18.6.tar.gz",
-    ],
-)
-
-http_archive(
-    name = "bazel_gazelle",
-    sha256 = "3c681998538231a2d24d0c07ed5a7658cb72bfb5fd4bf9911157c0e9ac6a2687",
-    urls = ["https://github.com/bazelbuild/bazel-gazelle/releases/download/0.17.0/bazel-gazelle-0.17.0.tar.gz"],
-)
-
-load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")
-
-go_rules_dependencies()
-
-go_register_toolchains()
-
-load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies", "go_repository")
-
-gazelle_dependencies()
-
 # Linux sources.
 
 http_archive(

Cheers,
Chris

@arunmastermind
Copy link
Author

Hi Chris,

Thanks for the advise and seems to have removed so many errors but still I am getting one as:

bazel run //deeplearning/clgen -- \ --config $PWD/deeplearning/clgen/tests/data/tiny/config.pbtxt
WARNING: Output base '/home/arunkumar/.cache/bazel/_bazel_arunkumar/fd846354cac299b536a7fc9db5b51c70' is on NFS. This may lead to surprising failures and undetermined behavior.
ERROR: error loading package '': Extension file not found. Unable to load file '@bazel_tools//tools/build_defs/repo:jvm.bzl': file doesn't exist or isn't a file
ERROR: error loading package '': Extension file not found. Unable to load file '@bazel_tools//tools/build_defs/repo:jvm.bzl': file doesn't exist or isn't a file
INFO: Elapsed time: 0.237s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)

I hope this is also something that's getting blank in the build.
Could you please advise something on this so that we can proceed further.

Regards
AK

@arunmastermind
Copy link
Author

Hi Chris,

Could you advise anything on this error?

Regards
AK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants