wildcard bug when matched file count is 1

[porscheme]

Importing files with wildcard in the path failed. It works okay, with exact file name.

Below is the error I'm getting

C:\nebulaimporter>docker run --rm -it --add-host host.docker.internal:host-gateway --network host -v C:\coldstart\pending_vertices\test.vertices.yaml:/coldstart/pending_vertices/test.vertices.yaml -v C:\coldstart:/coldstart vesoft/nebula-importer:v2.6.0 --config /coldstart/pending_vertices/test.vertices.yaml
2022/03/18 04:46:49 --- START OF NEBULA IMPORTER ---
2022/03/18 04:46:49 File(/coldstart/pending_vertices/students/*.CSV) doesn't exist
2022/03/18 04:46:50 --- END OF NEBULA IMPORTER ---

YAML FILE

version: v2
description: example
removeTempFiles: false
clientSettings:
  retry: 3
  concurrency: 1 # number of graph clients
  channelBufferSize: 1
  space: StudentCentral
  connection:
    user: root
    password: nebula
    address: host.docker.internal:9669
  postStart:
    commands: |
      DROP SPACE IF EXISTS StudentCentral;    
      CREATE SPACE IF NOT EXISTS StudentCentral(partition_num=6, replica_factor=2, vid_type=FIXED_STRING(80));
      USE StudentCentral;
      CREATE TAG IF NOT EXISTS                      Student(sudentId string, hcs string, docInstance string);
maritalStatusId int, raceIds string);
    afterPeriod: 8s
logPath: /csv_data/err/test.log
files:
  - path: ./students/*.CSV
    batchSize: 10000
    inOrder: false
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: ","
    schema:
      type: vertex
      vertex:
        vid:
          type: string
          index: 0
        tags:
          - name: Patient
            props:
              - name: sudentId
                type: string
              - name: hcs
                type: string
              - name: docInstance
                type: string

[porscheme]

@wey-gu can you please help me with this issue?

[wey-gu]

could you help with this?

run --rm -it --add-host host.docker.internal:host-gateway --network host -v C:\coldstart\pending_vertices\test.vertices.yaml:/coldstart/pending_vertices/test.vertices.yaml -v C:\coldstart:/coldstart vesoft/nebula-importer:v2.6.0 /bin/ls -l /coldstart/pending_vertices/students/*.CSV

Or, ensure those files are named *.CSV rather than *.csv ?

[porscheme]

• I renamed *.CSV to *.csv; not worked
• I tried this command and I can list the data files

C:\>docker run --entrypoint /bin/sh --rm -it --add-host host.docker.internal:host-gateway --network host -v c:\coldstart\pending_vertices\test.vertices.yaml:/coldstart/pending_vertices/test.vertices.yaml -v c:\coldstart:/coldstart vesoft/nebula-importer:v2.6.0
~ # ls /coldstart/pending_vertices/students/*.csv
/coldstart/pending_vertices/students/test.csv
~ #

[wey-gu]

Could you kindly help with this, to see if it's a bug only when the filename starts with *？

...
files:
-  - path: ./students/*.CSV
+  - path: ./students/t*.csv
    batchSize: 10000
    inOrder: false
    type: csv
    csv:
...

[porscheme]

fyi...I'm using v2.6.0 importer.

2022/03/18 07:11:13 --- START OF NEBULA IMPORTER ---
2022/03/18 07:11:13 File(/root/students/t*.csv) doesn't exist
2022/03/18 07:11:14 --- END OF NEBULA IMPORTER ---

[porscheme]

Upon further testing...

• If there is only one file at the target location, wildcard in the file path doesn't work
• If more than one file at the target location, wildcard in the file path does work fine

[wey-gu]

Thanks @porscheme , this should be a bug to be addressed, thanks!
will keep this opened.

[porscheme]

• How does this work in v3.0.1
• Does v3.0.1 supported in open source K8S cluster deployment? I'm asking this Q because, helm charts are still referring to v2.6.1

[wey-gu]

• This should be working on v3.0.1, too, with exactly the same importer config in v2.x
• v3.x nebula-operator is not yet supported now, will be supported later :)

[porscheme]

Thanks @wey-gu for the reply.

If I understand correctly what you said, v3.0.1 and beyond open-source deployment will supported, can you confirm?
We are correctly using v2.6.1 cluster; is there a way we can migrate to v3.0.1 right now?
I will be on the Nebula community later today at 12:00 PM, can you join I have few questions?

[wey-gu]

I will join later :-), see you there!

[wey-gu]

v3.0.1 and beyond open-source deployment will supported, can you confirm?

wey: yes, I confirm it will be supported

We are correctly using v2.6.1 cluster; is there a way we can migrate to v3.0.1 right now?

wey: For now, in k8s, it cannot be migrated, while if it's bearemetal deployed, the upgrade/migration progress is in docs :)

[wey-gu]

Thanks @wey-gu for the reply.

If I understand correctly what you said, v3.0.1 and beyond open-source deployment will supported, can you confirm? We are correctly using v2.6.1 cluster; is there a way we can migrate to v3.0.1 right now? I will be on the Nebula community later today at 12:00 PM, can you join I have few questions?

For the k8s of 3.x, @MegaByte875 will share his draft PR when it's a little bit more mature before merging, if you are interested you could help test it soon.

Thanks!

[veezhang]

@porscheme Thanks!
And it's already fixed in #210 .
And you can use the v2.6 branch .

[QingZ11]

I have noticed that the issue you created hasn’t been updated for nearly a month, so I have to close it for now. If you have any new updates, you are welcome to reopen this issue anytime.

Thanks a lot for your contribution anyway 😊