Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-31538: [Python][Docs] Documents ParquetWriteOptions class #38279

Closed
wants to merge 11 commits into from
Closed

GH-31538: [Python][Docs] Documents ParquetWriteOptions class #38279

wants to merge 11 commits into from

Conversation

Divyansh200102
Copy link
Contributor

@Divyansh200102 Divyansh200102 commented Oct 16, 2023 •
edited by github-actions bot
Loading

@Divyansh200102 Divyansh200102 changed the title GH-31538:[Python][Docs] Documents ParquetWriteOptions class MINOR:[Python][Docs] Documents ParquetWriteOptions class Oct 16, 2023
@github-actions github-actions bot added the awaiting review Awaiting review label Oct 16, 2023
@github-actions
Copy link

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

In the case of PARQUET issues on JIRA the title also supports:

PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

See also:

@AlenkaF AlenkaF changed the title MINOR:[Python][Docs] Documents ParquetWriteOptions class GH-31538:[Python][Docs] Documents ParquetWriteOptions class Oct 18, 2023
@AlenkaF
Copy link
Member

AlenkaF commented Oct 18, 2023

Thank you for the contribution @Divyansh200102 !

I think the issue was about adding docstrings to ParquetFileWriteOptions class, similar to ParquetReadOptions and then adding ParquetFileWriteOptions to API docs, not ParquetWriteOptions.

@AlenkaF
Copy link
Member

AlenkaF commented Oct 24, 2023

Thanks!

To fix the CI the indentation of the docstrings has to match. Also, can we update the examples to use a structure we use in other docstrings. For example in dataset.scanner:

arrow/python/pyarrow/_dataset.pyx

Lines 347 to 392 in c8c7485

Examples
--------
>>> import pyarrow as pa
>>> table = pa.table({'year': [2020, 2022, 2021, 2022, 2019, 2021],
... 'n_legs': [2, 2, 4, 4, 5, 100],
... 'animal': ["Flamingo", "Parrot", "Dog", "Horse",
... "Brittle stars", "Centipede"]})
>>>
>>> import pyarrow.parquet as pq
>>> pq.write_table(table, "dataset_scanner.parquet")
>>> import pyarrow.dataset as ds
>>> dataset = ds.dataset("dataset_scanner.parquet")
Selecting a subset of the columns:
>>> dataset.scanner(columns=["year", "n_legs"]).to_table()
pyarrow.Table
year: int64
n_legs: int64
----
year: [[2020,2022,2021,2022,2019,2021]]
n_legs: [[2,2,4,4,5,100]]
Projecting selected columns using an expression:
>>> dataset.scanner(columns={
... "n_legs_uint": ds.field("n_legs").cast("uint8"),
... }).to_table()
pyarrow.Table
n_legs_uint: uint8
----
n_legs_uint: [[2,2,4,4,5,100]]
Filtering rows while scanning:
>>> dataset.scanner(filter=ds.field("year") > 2020).to_table()
pyarrow.Table
year: int64
n_legs: int64
animal: string
----
year: [[2022,2021,2022,2021]]
n_legs: [[2,4,4,100]]
animal: [["Parrot","Dog","Horse","Centipede"]]
"""

@Divyansh200102 Divyansh200102 reopened this Feb 1, 2024
@Divyansh200102
Copy link
Contributor Author

Divyansh200102 commented Feb 17, 2024 •
edited
Loading

@AlenkaF I am trying to build the _dataset.pyx using https://arrow.apache.org/docs/developers/python.html#doctest to find the errors but the below error is coming while building.Could you please take a look at the below log? Thank you for your patience.

Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub
$ pushd arrow/python
~/Documents/GitHub/arrow/python ~/Documents/GitHub

Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub/arrow/python (main)
$ python -m venv myenv

Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub/arrow/python (main)
$ source myenv/Scripts/activate
(myenv) 
Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub/arrow/python (main)
$ pip install pytest-cython
Collecting pytest-cython
  Obtaining dependency information for pytest-cython from https://files.pythonhosted.org/packages/20/e6/dd6f6fae2e4e15f2abc5d95da945d5044bb2a4488edc8bd9c8c6a8645376/pytest_cython-0.2.1-py2.py3-none-any.whl.metadata
  Using cached pytest_cython-0.2.1-py2.py3-none-any.whl.metadata (3.4 kB)
Collecting pytest>=4.6.0 (from pytest-cython)
  Obtaining dependency information for pytest>=4.6.0 from https://files.pythonhosted.org/packages/2e/28/30125a808a2448d72fdba26d01ef2bec76a3c860c8694b636e6104e38713/pytest-8.0.1-py3-none-any.whl.metadata
  Using cached pytest-8.0.1-py3-none-any.whl.metadata (7.7 kB)
Collecting iniconfig (from pytest>=4.6.0->pytest-cython)
  Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting packaging (from pytest>=4.6.0->pytest-cython)
  Obtaining dependency information for packaging from https://files.pythonhosted.org/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl.metadata
  Using cached packaging-23.2-py3-none-any.whl.metadata (3.2 kB)
Collecting pluggy<2.0,>=1.3.0 (from pytest>=4.6.0->pytest-cython)
  Obtaining dependency information for pluggy<2.0,>=1.3.0 from https://files.pythonhosted.org/packages/a5/5b/0cc789b59e8cc1bf288b38111d002d8c5917123194d45b29dcdac64723cc/pluggy-1.4.0-py3-none-any.whl.metadata
  Using cached pluggy-1.4.0-py3-none-any.whl.metadata (4.3 kB)
Collecting colorama (from pytest>=4.6.0->pytest-cython)
  Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Using cached pytest_cython-0.2.1-py2.py3-none-any.whl (6.4 kB)
Using cached pytest-8.0.1-py3-none-any.whl (333 kB)
Using cached pluggy-1.4.0-py3-none-any.whl (20 kB)
Using cached packaging-23.2-py3-none-any.whl (53 kB)
Installing collected packages: pluggy, packaging, iniconfig, colorama, pytest, pytest-cython
Successfully installed colorama-0.4.6 iniconfig-2.0.0 packaging-23.2 pluggy-1.4.0 pytest-8.0.1 pytest-cython-0.2.1

[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
(myenv) 
Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub/arrow/python (main)
$ pytest --doctest-cython
C:\Users\Divyansh\Documents\GitHub\arrow\python\myenv\Lib\site-packages\pytest_cython\__init__.py:7: UserWarning: could not get pytest-cython version; pkg_resources not available or package not installed
  warnings.warn('could not get pytest-cython version; pkg_resources '
=============================================================================== test session starts ===============================================================================
platform win32 -- Python 3.12.0, pytest-8.0.1, pluggy-1.4.0
rootdir: C:\Users\Divyansh\Documents\GitHub\arrow\python
configfile: setup.cfg
plugins: cython-0.2.1
collected 0 items / 2 errors                                                                                                                                                       

===================================================================================== ERRORS ====================================================================================== 
____________________________________________________________________________ ERROR collecting pyarrow _____________________________________________________________________________ 
C:\Python312\Lib\importlib\__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1381: in _gcd_import
    ???
<frozen importlib._bootstrap>:1354: in _find_and_load
    ???
<frozen importlib._bootstrap>:1304: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:488: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1381: in _gcd_import
    ???
<frozen importlib._bootstrap>:1354: in _find_and_load
    ???
<frozen importlib._bootstrap>:1325: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:929: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:994: in exec_module
    ???
<frozen importlib._bootstrap>:488: in _call_with_frames_removed
    ???
pyarrow\__init__.py:65: in <module>
    import pyarrow.lib as _lib
E   ModuleNotFoundError: No module named 'pyarrow.lib'
_________________________________________________ ERROR collecting pythonenv/Lib/site-packages/numpy/testing/tests/test_utils.py __________________________________________________ 
ImportError while importing test module 'C:\Users\Divyansh\Documents\GitHub\arrow\python\pythonenv\Lib\site-packages\numpy\testing\tests\test_utils.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Python312\Lib\importlib\__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pythonenv\Lib\site-packages\numpy\testing\__init__.py:11: in <module>
    from ._private.utils import *
pythonenv\Lib\site-packages\numpy\testing\_private\utils.py:21: in <module>
    import numpy as np
E   ModuleNotFoundError: No module named 'numpy'
============================================================================= short test summary info ============================================================================= 
ERROR pyarrow - ModuleNotFoundError: No module named 'pyarrow.lib'
ERROR pythonenv/Lib/site-packages/numpy/testing/tests/test_utils.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 
================================================================================ 2 errors in 0.32s ================================================================================ 
(myenv) 
Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub/arrow/python (main)
$ pip install pyarrow
Collecting pyarrow
  Obtaining dependency information for pyarrow from https://files.pythonhosted.org/packages/1a/f7/f6df7992ef2339bbf31ba349de19af5b8fd75590129c4e8fcb719f24fe5f/pyarrow-15.0.0-cp312-cp312-win_amd64.whl.metadata
  Using cached pyarrow-15.0.0-cp312-cp312-win_amd64.whl.metadata (3.1 kB)
Collecting numpy<2,>=1.16.6 (from pyarrow)
  Obtaining dependency information for numpy<2,>=1.16.6 from https://files.pythonhosted.org/packages/16/2e/86f24451c2d530c88daf997cb8d6ac622c1d40d19f5a031ed68a4b73a374/numpy-1.26.4-cp312-cp312-win_amd64.whl.metadata
  Using cached numpy-1.26.4-cp312-cp312-win_amd64.whl.metadata (61 kB)
Using cached pyarrow-15.0.0-cp312-cp312-win_amd64.whl (25.3 MB)
Using cached numpy-1.26.4-cp312-cp312-win_amd64.whl (15.5 MB)
Installing collected packages: numpy, pyarrow
Successfully installed numpy-1.26.4 pyarrow-15.0.0

[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
(myenv) 
Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub/arrow/python (main)
$ python -m pytest --doctest-cython
C:\Users\Divyansh\Documents\GitHub\arrow\python\myenv\Lib\site-packages\pytest_cython\__init__.py:7: UserWarning: could not get pytest-cython version; pkg_resources not available or package not installed
  warnings.warn('could not get pytest-cython version; pkg_resources '
=============================================================================== test session starts ===============================================================================
platform win32 -- Python 3.12.0, pytest-8.0.1, pluggy-1.4.0
rootdir: C:\Users\Divyansh\Documents\GitHub\arrow\python
configfile: setup.cfg
plugins: cython-0.2.1
collected 139 items / 1 error                                                                                                                                                      

===================================================================================== ERRORS ====================================================================================== 
____________________________________________________________________________ ERROR collecting pyarrow _____________________________________________________________________________ 
C:\Python312\Lib\importlib\__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1381: in _gcd_import
    ???
<frozen importlib._bootstrap>:1354: in _find_and_load
    ???
<frozen importlib._bootstrap>:1304: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:488: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1381: in _gcd_import
    ???
<frozen importlib._bootstrap>:1354: in _find_and_load
    ???
<frozen importlib._bootstrap>:1325: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:929: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:994: in exec_module
    ???
<frozen importlib._bootstrap>:488: in _call_with_frames_removed
    ???
pyarrow\__init__.py:65: in <module>
    import pyarrow.lib as _lib
E   ModuleNotFoundError: No module named 'pyarrow.lib'
================================================================================ warnings summary ================================================================================= 
pythonenv\Lib\site-packages\numpy\testing\tests\test_utils.py:1585
  C:\Users\Divyansh\Documents\GitHub\arrow\python\pythonenv\Lib\site-packages\numpy\testing\tests\test_utils.py:1585: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.slow

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================================= short test summary info ============================================================================= 
ERROR pyarrow - ModuleNotFoundError: No module named 'pyarrow.lib'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 
=========================================================================== 1 warning, 1 error in 0.53s ===========================================================================

@AlenkaF
Copy link
Member

AlenkaF commented Feb 19, 2024

Hm, not sure but first I need to ask if PyArrow is built? To have _dataset.pyx built you need to follow these instructions: https://arrow.apache.org/docs/developers/python.html#building-on-linux-and-macos and then you can run doctest.

@Divyansh200102
Copy link
Contributor Author

Hm, not sure but first I need to ask if PyArrow is built? To have _dataset.pyx built you need to follow these instructions: https://arrow.apache.org/docs/developers/python.html#building-on-linux-and-macos and then you can run doctest.

@AlenkaF For some reason I am not able to build the documentation and it's taking a lot of time to figure it out so if time allows it could you please run the preview-docs command and suggest me the changes to be made? It would be a great help.

@AlenkaF
Copy link
Member

AlenkaF commented Feb 22, 2024

Sure!

I would suggest to first fix the CI:

--- original//arrow/python/pyarrow/_dataset_parquet.pyx
+++ fixed//arrow/python/pyarrow/_dataset_parquet.pyx
@@ -626,12 +626,12 @@
         >>> options = pq.ParquetFileWriteOptions(compression_level=5)
         >>> pq.write_table(table, "example_compression.parquet", write_options=options)
 
-   
+
     def update(self, **kwargs):
         """
         Parameters
         ----------
-        **kwargs : dict
+        **kwargs: dict
         """
         arrow_fields = {
             "use_deprecated_int96_timestamps",

Regarding the preview docs you can run the command yourself whenever you need. Simply add @github-actions crossbow submit preview-docs to the comment, see https://arrow.apache.org/docs/dev/developers/documentation.html#building-a-docs-preview-in-a-pull-request

@Divyansh200102 Divyansh200102 force-pushed the Document-ParquetWriteOptions-class branch from ce62fbc to a7abce0 Compare February 22, 2024 08:01
@Divyansh200102
Copy link
Contributor Author

@github-actions crossbow submit preview-docs

@github-actions GitHub Actions
Copy link 

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/8001313647

@Divyansh200102
Copy link
Contributor Author

@AlenkaF is there a reason why this is coming

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/8001313647

@AlenkaF
Copy link
Member

AlenkaF commented Feb 22, 2024

@github-actions crossbow submit preview-docs

@github-actions GitHub Actions
Copy link

Revision: a7abce0

Submitted crossbow builds: ursacomputing/crossbow @ actions-c547c944ea

Task Status
preview-docs GitHub Actions

@AlenkaF
Copy link
Member

AlenkaF commented Feb 22, 2024

Thought the preview-docs bot works for everybody, @assignUser is docs-preview crossbow job enabled only for the contributors?

In any case, the preview will not be successful as there are still errors to be fixed. The linter error I mentioned in #38279 (comment) plus a missing """ at the end of the docstrings added in this PR. See

  Error compiling Cython file:
  ------------------------------------------------------------
  ...


      def update(self, **kwargs):
          """
          Parameters
          ----------
                    ^
  ------------------------------------------------------------```

@Divyansh200102
Copy link
Contributor Author

Divyansh200102 commented Feb 22, 2024 •
edited
Loading

I am not sure why these files are getting formatted array.pxi,table.pxi,types.pxi I don't know if this is an error from my compiler? Either way the checks were still failing.I am using these docs for fixing linter error.I am very sorry for troubling you with such small troubles but I facing these formatting issues another 2PR's I am working on and it's a bit frustrating to not being able to figure out the cause.In one pr my docker-compose.yml file is getting unnecessary errors and in another pr the same files as mentioned above are getting formatted for no reason(#40186).So could you please suggest me something to solve this issue it would be a great help @AlenkaF. Thank you for your patience
cc @kou

diff --git a/python/pyarrow/_dataset_parquet.pyx b/python/pyarrow/_dataset_parquet.pyx
index 2309c705d..e24fcf9d2 100644
--- a/python/pyarrow/_dataset_parquet.pyx
+++ b/python/pyarrow/_dataset_parquet.pyx
@@ -626,7 +626,7 @@ cdef class ParquetFileWriteOptions(FileWriteOptions):
         >>> options = pq.ParquetFileWriteOptions(compression_level=5)
         >>> pq.write_table(table, "example_compression.parquet", write_options=options)
         """
-   
+
     def update(self, **kwargs):
         """
         Parameters
diff --git a/python/pyarrow/array.pxi b/python/pyarrow/array.pxi
index ad01d4557..e441278ae 100644
--- a/python/pyarrow/array.pxi
+++ b/python/pyarrow/array.pxi
@@ -1748,7 +1748,8 @@ cdef class Array(_PandasConvertible):
                     inner_array = pyarrow_unwrap_array(casted_array)
                 except ArrowInvalid as e:
                     raise ValueError(
-                        f"Could not cast {self.type} to requested type {target_type}: {e}"
+                        f"Could not cast {self.type} to requested type {
+                            target_type}: {e}"
                     )
             else:
                 inner_array = self.sp_array
diff --git a/python/pyarrow/table.pxi b/python/pyarrow/table.pxi
index ee3872aa3..18b7944d1 100644
--- a/python/pyarrow/table.pxi
+++ b/python/pyarrow/table.pxi
@@ -3081,7 +3081,8 @@ cdef class RecordBatch(_Tabular):
                     inner_batch = pyarrow_unwrap_batch(casted_batch)
                 except ArrowInvalid as e:
                     raise ValueError(
-                        f"Could not cast {self.schema} to requested schema {target_schema}: {e}"
+                        f"Could not cast {self.schema} to requested schema {
+                            target_schema}: {e}"
                     )
             else:
                 inner_batch = self.sp_batch
diff --git a/python/pyarrow/types.pxi b/python/pyarrow/types.pxi
index e9bf56c62..1f7773174 100644
--- a/python/pyarrow/types.pxi
+++ b/python/pyarrow/types.pxi
@@ -133,7 +133,8 @@ cdef void* _as_c_pointer(v, allow_null=False) except *:
         else:
             capsule_name_str = capsule_name.decode()
             raise ValueError(
-                f"Can't convert PyCapsule with name '{capsule_name_str}' to pointer address"
+                f"Can't convert PyCapsule with name '{
+                    capsule_name_str}' to pointer address"
             )
     else:
         raise TypeError(f"Expected a pointer value, got {type(v)!r}")

@AlenkaF
Copy link
Member

AlenkaF commented Feb 22, 2024

I am not sure why these files are getting formattted array.pxi,table.pxi,types.pxi I don't know if this is an error from my compiler? Either way the checks were still failing.

They are still failing but the error is different now. Be sure to clean any whitespace in the "empty" line. It looks like they are left after you added """.

I am using these docs for fixing linter error.I am very sorry for troubling you with such seemingly small troubles but I facing these formatting issues another 2PR's I am working on and it's a bit frustrraing to not being able to figure out the cause.In one pr my docker-compose.yml file is getting unnecessary errors and in another pr the same files as mentioned above are getting formatted for no reason(#40186).

You are using the Archery tool? Hm, maybe try to reinstall it?

@Divyansh200102 Divyansh200102 mentioned this pull request Feb 22, 2024
Closed
@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Feb 22, 2024
@assignUser
Copy link
Member

is docs-preview crossbow job enabled only for the contributors?

It's not specific to the docs-preview job. We recently tighten the permissions a bit for security reasons, you now have to be a committer or collaborator to run jobs.

@kou kou changed the title GH-31538:[Python][Docs] Documents ParquetWriteOptions class GH-31538: [Python][Docs] Documents ParquetWriteOptions class Feb 22, 2024
@github-actions GitHub Actions
Copy link

⚠️ GitHub issue #31538 has been automatically assigned in GitHub to PR creator.

@kou
Copy link
Member

kou commented Feb 22, 2024

Could you show the exact command line your used for formatting?

@Divyansh200102
Copy link
Contributor Author

Divyansh200102 commented Feb 24, 2024 •
edited
Loading

Could you show the exact command line your used for formatting?

Sure @kou

I basically first created a virtual env and then used pip install -e "arrow/dev/archery[lint]" and then this cmd archery lint --python --clang-format --fix

Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub/arrow (Document-ParquetWriteOptions-class)
$ source myenv/Scripts/activate
(myenv) 
Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub/arrow (Document-ParquetWriteOptions-class)
$ cd ..
(myenv) 
Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub
$ pip install -e "arrow/dev/archery[lint]"
Obtaining file:///C:/Users/Divyansh/Documents/GitHub/arrow/dev/archery
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting click>=7 (from archery==0.1.0)
  Obtaining dependency information for click>=7 from https://files.pythonhosted.org/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl.metadata
  Using cached click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting numpydoc==1.1.0 (from archery==0.1.0)
  Using cached numpydoc-1.1.0-py3-none-any.whl (47 kB)
Collecting autopep8 (from archery==0.1.0)
  Obtaining dependency information for autopep8 from https://files.pythonhosted.org/packages/d8/f2/e63c9f9c485cd90df8e4e7ae90fa3be2469c9641888558c7b45fa98a76f8/autopep8-2.0.4-py2.py3-none-any.whl.metadata
  Using cached autopep8-2.0.4-py2.py3-none-any.whl.metadata (17 kB)
Collecting flake8==6.1.0 (from archery==0.1.0)
  Obtaining dependency information for flake8==6.1.0 from https://files.pythonhosted.org/packages/b0/24/bbf7175ffc47cb3d3e1eb523ddb23272968359dfcf2e1294707a2bf12fc4/flake8-6.1.0-py2.py3-none-any.whl.metadata
  Using cached flake8-6.1.0-py2.py3-none-any.whl.metadata (3.8 kB)
Collecting cython-lint (from archery==0.1.0)
  Obtaining dependency information for cython-lint from https://files.pythonhosted.org/packages/02/5d/2e8f5cec532b056fbc644b680b132abcb64a73d3a5196e3de797616b392e/cython_lint-0.16.0-py2.py3-none-any.whl.metadata
  Using cached cython_lint-0.16.0-py2.py3-none-any.whl.metadata (4.8 kB)
Collecting cmake-format==0.6.13 (from archery==0.1.0)
  Using cached cmake_format-0.6.13-py3-none-any.whl (19 kB)
Collecting cmakelang>=0.6.13 (from cmake-format==0.6.13->archery==0.1.0)
  Using cached cmakelang-0.6.13-py3-none-any.whl (159 kB)
Collecting mccabe<0.8.0,>=0.7.0 (from flake8==6.1.0->archery==0.1.0)
  Obtaining dependency information for mccabe<0.8.0,>=0.7.0 from https://files.pythonhosted.org/packages/27/1a/1f68f9ba0c207934b35b86a8ca3aad8395a3d6dd7921c0686e23853ff5a9/mccabe-0.7.0-py2.py3-none-any.whl.metadata
  Downloading mccabe-0.7.0-py2.py3-none-any.whl.metadata (5.0 kB)
Collecting pycodestyle<2.12.0,>=2.11.0 (from flake8==6.1.0->archery==0.1.0)
  Obtaining dependency information for pycodestyle<2.12.0,>=2.11.0 from https://files.pythonhosted.org/packages/b1/90/a998c550d0ddd07e38605bb5c455d00fcc177a800ff9cc3dafdcb3dd7b56/pycodestyle-2.11.1-py2.py3-none-any.whl.metadata
  Using cached pycodestyle-2.11.1-py2.py3-none-any.whl.metadata (4.5 kB)
Collecting pyflakes<3.2.0,>=3.1.0 (from flake8==6.1.0->archery==0.1.0)
  Obtaining dependency information for pyflakes<3.2.0,>=3.1.0 from https://files.pythonhosted.org/packages/00/e9/1e1fd7fae559bfd07704991e9a59dd1349b72423c904256c073ce88a9940/pyflakes-3.1.0-py2.py3-none-any.whl.metadata
  Using cached pyflakes-3.1.0-py2.py3-none-any.whl.metadata (3.5 kB)
Collecting Jinja2>=2.3 (from numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for Jinja2>=2.3 from https://files.pythonhosted.org/packages/30/6d/6de6be2d02603ab56e72997708809e8a5b0fbfee080735109b40a3564843/Jinja2-3.1.3-py3-none-any.whl.metadata
  Using cached Jinja2-3.1.3-py3-none-any.whl.metadata (3.3 kB)
Collecting sphinx>=1.6.5 (from numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for sphinx>=1.6.5 from https://files.pythonhosted.org/packages/b2/b6/8ed35256aa530a9d3da15d20bdc0ba888d5364441bb50a5a83ee7827affe/sphinx-7.2.6-py3-none-any.whl.metadata
  Using cached sphinx-7.2.6-py3-none-any.whl.metadata (5.9 kB)
Collecting colorama (from click>=7->archery==0.1.0)
  Obtaining dependency information for colorama from https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl.metadata
  Using cached colorama-0.4.6-py2.py3-none-any.whl.metadata (17 kB)
Collecting cython>=0.29.32 (from cython-lint->archery==0.1.0)
  Obtaining dependency information for cython>=0.29.32 from https://files.pythonhosted.org/packages/b4/69/cb4620287cd9ef461103e122c0a2ae7f7ecf183e02510676fb5a15c95b05/Cython-3.0.8-cp312-cp312-win_amd64.whl.metadata
  Using cached Cython-3.0.8-cp312-cp312-win_amd64.whl.metadata (3.2 kB)
Collecting tokenize-rt>=3.2.0 (from cython-lint->archery==0.1.0)
  Obtaining dependency information for tokenize-rt>=3.2.0 from https://files.pythonhosted.org/packages/8d/35/78f03aa48cfebd13646707f64477bc7eacf1081edcdcd1b4d57cb1b5d0a8/tokenize_rt-5.2.0-py2.py3-none-any.whl.metadata
  Using cached tokenize_rt-5.2.0-py2.py3-none-any.whl.metadata (4.1 kB)
Collecting six>=1.13.0 (from cmakelang>=0.6.13->cmake-format==0.6.13->archery==0.1.0)
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting MarkupSafe>=2.0 (from Jinja2>=2.3->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for MarkupSafe>=2.0 from https://files.pythonhosted.org/packages/3f/14/c3554d512d5f9100a95e737502f4a2323a1959f6d0d01e0d0997b35f7b10/MarkupSafe-2.1.5-cp312-cp312-win_amd64.whl.metadata
  Using cached MarkupSafe-2.1.5-cp312-cp312-win_amd64.whl.metadata (3.1 kB)
Collecting sphinxcontrib-applehelp (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for sphinxcontrib-applehelp from https://files.pythonhosted.org/packages/56/89/fea3fbf6785b388e6cb8a1beaf62f96e80b37311bdeed6e133388a732426/sphinxcontrib_applehelp-1.0.8-py3-none-any.whl.metadata
  Using cached sphinxcontrib_applehelp-1.0.8-py3-none-any.whl.metadata (2.3 kB)
Collecting sphinxcontrib-devhelp (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for sphinxcontrib-devhelp from https://files.pythonhosted.org/packages/a0/52/1049d918d1d1c72857d285c3f0c64c1cbe0be394ce1c93a3d2aa4f39fe3b/sphinxcontrib_devhelp-1.0.6-py3-none-any.whl.metadata
  Using cached sphinxcontrib_devhelp-1.0.6-py3-none-any.whl.metadata (2.3 kB)
Collecting sphinxcontrib-jsmath (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Using cached sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl (5.1 kB)
Collecting sphinxcontrib-htmlhelp>=2.0.0 (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for sphinxcontrib-htmlhelp>=2.0.0 from https://files.pythonhosted.org/packages/c2/e9/74c4cda5b409af3222fda38f0774e616011bc935f639dbc0da5ca2d1be7d/sphinxcontrib_htmlhelp-2.0.5-py3-none-any.whl.metadata
  Using cached sphinxcontrib_htmlhelp-2.0.5-py3-none-any.whl.metadata (2.3 kB)
Collecting sphinxcontrib-serializinghtml>=1.1.9 (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for sphinxcontrib-serializinghtml>=1.1.9 from https://files.pythonhosted.org/packages/38/24/228bb903ea87b9e08ab33470e6102402a644127108c7117ac9c00d849f82/sphinxcontrib_serializinghtml-1.1.10-py3-none-any.whl.metadata
  Using cached sphinxcontrib_serializinghtml-1.1.10-py3-none-any.whl.metadata (2.4 kB)
Collecting sphinxcontrib-qthelp (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for sphinxcontrib-qthelp from https://files.pythonhosted.org/packages/80/b3/1beac14a88654d2e5120d0143b49be5ad450b86eb1963523d8dbdcc51eb2/sphinxcontrib_qthelp-1.0.7-py3-none-any.whl.metadata
  Using cached sphinxcontrib_qthelp-1.0.7-py3-none-any.whl.metadata (2.2 kB)
Collecting Pygments>=2.14 (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for Pygments>=2.14 from https://files.pythonhosted.org/packages/97/9c/372fef8377a6e340b1704768d20daaded98bf13282b5327beb2e2fe2c7ef/pygments-2.17.2-py3-none-any.whl.metadata
  Using cached pygments-2.17.2-py3-none-any.whl.metadata (2.6 kB)
Collecting docutils<0.21,>=0.18.1 (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for docutils<0.21,>=0.18.1 from https://files.pythonhosted.org/packages/26/87/f238c0670b94533ac0353a4e2a1a771a0cc73277b88bff23d3ae35a256c1/docutils-0.20.1-py3-none-any.whl.metadata
  Using cached docutils-0.20.1-py3-none-any.whl.metadata (2.8 kB)
Collecting snowballstemmer>=2.0 (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Using cached snowballstemmer-2.2.0-py2.py3-none-any.whl (93 kB)
Collecting babel>=2.9 (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for babel>=2.9 from https://files.pythonhosted.org/packages/0d/35/4196b21041e29a42dc4f05866d0c94fa26c9da88ce12c38c2265e42c82fb/Babel-2.14.0-py3-none-any.whl.metadata
  Using cached Babel-2.14.0-py3-none-any.whl.metadata (1.6 kB)
Collecting alabaster<0.8,>=0.7 (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for alabaster<0.8,>=0.7 from https://files.pythonhosted.org/packages/32/34/d4e1c02d3bee589efb5dfa17f88ea08bdb3e3eac12bc475462aec52ed223/alabaster-0.7.16-py3-none-any.whl.metadata
  Using cached alabaster-0.7.16-py3-none-any.whl.metadata (2.9 kB)
Collecting imagesize>=1.3 (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for imagesize>=1.3 from https://files.pythonhosted.org/packages/ff/62/85c4c919272577931d407be5ba5d71c20f0b616d31a0befe0ae45bb79abd/imagesize-1.4.1-py2.py3-none-any.whl.metadata
  Using cached imagesize-1.4.1-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting requests>=2.25.0 (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for requests>=2.25.0 from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting packaging>=21.0 (from sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for packaging>=21.0 from https://files.pythonhosted.org/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl.metadata
  Using cached packaging-23.2-py3-none-any.whl.metadata (3.2 kB)
Collecting charset-normalizer<4,>=2 (from requests>=2.25.0->sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/b6/7c/8debebb4f90174074b827c63242c23851bdf00a532489fba57fef3416e40/charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl.metadata
  Using cached charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl.metadata (34 kB)
Collecting idna<4,>=2.5 (from requests>=2.25.0->sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for idna<4,>=2.5 from https://files.pythonhosted.org/packages/c2/e7/a82b05cf63a603df6e68d59ae6a68bf5064484a0718ea5033660af4b54a9/idna-3.6-py3-none-any.whl.metadata
  Using cached idna-3.6-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests>=2.25.0->sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/a2/73/a68704750a7679d0b6d3ad7aa8d4da8e14e151ae82e6fee774e6e0d05ec8/urllib3-2.2.1-py3-none-any.whl.metadata
  Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests>=2.25.0->sphinx>=1.6.5->numpydoc==1.1.0->archery==0.1.0)
  Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/ba/06/a07f096c664aeb9f01624f858c3add0a4e913d6c96257acb4fce61e7de14/certifi-2024.2.2-py3-none-any.whl.metadata
  Using cached certifi-2024.2.2-py3-none-any.whl.metadata (2.2 kB)
Using cached flake8-6.1.0-py2.py3-none-any.whl (58 kB)
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Using cached autopep8-2.0.4-py2.py3-none-any.whl (45 kB)
Using cached cython_lint-0.16.0-py2.py3-none-any.whl (12 kB)
Using cached Cython-3.0.8-cp312-cp312-win_amd64.whl (2.8 MB)
Using cached Jinja2-3.1.3-py3-none-any.whl (133 kB)
Using cached mccabe-0.7.0-py2.py3-none-any.whl (7.3 kB)
Using cached pycodestyle-2.11.1-py2.py3-none-any.whl (31 kB)
Using cached pyflakes-3.1.0-py2.py3-none-any.whl (62 kB)
Using cached sphinx-7.2.6-py3-none-any.whl (3.2 MB)
Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Using cached tokenize_rt-5.2.0-py2.py3-none-any.whl (5.8 kB)
Using cached alabaster-0.7.16-py3-none-any.whl (13 kB)
Using cached Babel-2.14.0-py3-none-any.whl (11.0 MB)
Using cached docutils-0.20.1-py3-none-any.whl (572 kB)
Using cached imagesize-1.4.1-py2.py3-none-any.whl (8.8 kB)
Using cached MarkupSafe-2.1.5-cp312-cp312-win_amd64.whl (17 kB)
Using cached packaging-23.2-py3-none-any.whl (53 kB)
Using cached pygments-2.17.2-py3-none-any.whl (1.2 MB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached sphinxcontrib_htmlhelp-2.0.5-py3-none-any.whl (99 kB)
Using cached sphinxcontrib_serializinghtml-1.1.10-py3-none-any.whl (92 kB)
Using cached sphinxcontrib_applehelp-1.0.8-py3-none-any.whl (120 kB)
Using cached sphinxcontrib_devhelp-1.0.6-py3-none-any.whl (83 kB)
Using cached sphinxcontrib_qthelp-1.0.7-py3-none-any.whl (89 kB)
Using cached certifi-2024.2.2-py3-none-any.whl (163 kB)
Using cached charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl (100 kB)
Using cached idna-3.6-py3-none-any.whl (61 kB)
Using cached urllib3-2.2.1-py3-none-any.whl (121 kB)
Building wheels for collected packages: archery
  Building editable for archery (pyproject.toml) ... done
  Created wheel for archery: filename=archery-0.1.0-0.editable-py3-none-any.whl size=3265 sha256=649b769ec42885fb97e90d4cd7ebc92ed4c81b71b3b55f76b8b375b3feec07ff
  Stored in directory: C:\Users\Divyansh\AppData\Local\Temp\pip-ephem-wheel-cache-gpmqil5u\wheels\c8\20\0b\32bf286b59e44a2eec44dacd43807c1f9bcc25107cdf0b860f
Successfully built archery
Installing collected packages: snowballstemmer, urllib3, tokenize-rt, sphinxcontrib-serializinghtml, sphinxcontrib-qthelp, sphinxcontrib-jsmath, sphinxcontrib-htmlhelp, sphinxcontr
ib-devhelp, sphinxcontrib-applehelp, six, Pygments, pyflakes, pycodestyle, packaging, mccabe, MarkupSafe, imagesize, idna, docutils, cython, colorama, charset-normalizer, certifi, babel, alabaster, requests, Jinja2, flake8, cython-lint, cmakelang, click, autopep8, sphinx, cmake-format, archery, numpydoc
Successfully installed Jinja2-3.1.3 MarkupSafe-2.1.5 Pygments-2.17.2 alabaster-0.7.16 archery-0.1.0 autopep8-2.0.4 babel-2.14.0 certifi-2024.2.2 charset-normalizer-3.3.2 click-8.1.
7 cmake-format-0.6.13 cmakelang-0.6.13 colorama-0.4.6 cython-3.0.8 cython-lint-0.16.0 docutils-0.20.1 flake8-6.1.0 idna-3.6 imagesize-1.4.1 mccabe-0.7.0 numpydoc-1.1.0 packaging-23
.2 pycodestyle-2.11.1 pyflakes-3.1.0 requests-2.31.0 six-1.16.0 snowballstemmer-2.2.0 sphinx-7.2.6 sphinxcontrib-applehelp-1.0.8 sphinxcontrib-devhelp-1.0.6 sphinxcontrib-htmlhelp-2.0.5 sphinxcontrib-jsmath-1.0.1 sphinxcontrib-qthelp-1.0.7 sphinxcontrib-serializinghtml-1.1.10 tokenize-rt-5.2.0 urllib3-2.2.1

[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
(myenv) 
Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub
$ archery lint --python --clang-format --fix
fatal: not a git repository (or any of the parent directories): .git
INFO:archery:Running C++ linters
-- Building using CMake version: 3.28.0-rc1
-- The C compiler identification is GNU 13.1.0
-- The CXX compiler identification is GNU 13.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/msys64/ucrt64/bin/cc.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/msys64/ucrt64/bin/c++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Arrow version: 16.0.0 (full: '16.0.0-SNAPSHOT')
-- Arrow SO version: 1600 (full: 1600.0.0)
-- clang-tidy 14 not found
-- clang-format 14 not found
-- Could NOT find ClangTools (missing: CLANG_FORMAT_BIN CLANG_TIDY_BIN)
-- infer not found
-- Found Python3: C:/Users/Divyansh/Documents/GitHub/arrow/myenv/Scripts/python.exe (found version "3.12.0") found components: Interpreter 
-- Found cpplint executable at C:/Users/Divyansh/Documents/GitHub/arrow/cpp/build-support/cpplint.py
ARROW_LINT_ONLY was specified, this is only a partial build directory
-- Configuring done (16.6s)
-- Generating done (0.0s)
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_EXPORT_COMPILE_COMMANDS
    CMAKE_UNITY_BUILD


-- Build files have been written to: C:/Users/Divyansh/AppData/Local/Temp/arrow-lint-g_7q80h8/cpp-build
make: *** No rule to make target 'format'.  Stop.
INFO:archery:Running Python formatter (autopep8)
INFO:archery:Running Python linter (flake8)
INFO:archery:Running Cython linter (cython-lint)
Skipping file C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\lib.pyx, as it cannot be parsed. Error: AttributeError("'_thread._local' object has no attribute 'cython_errors_stack'")
INFO:archery:Running clang-format for python/pyarrow/src/arrow/python
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\api.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\arrow_to_pandas.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\arrow_to_pandas.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\arrow_to_python_internal.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\async.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\benchmark.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\benchmark.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\common.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\common.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\csv.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\csv.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\datetime.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\datetime.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\decimal.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\decimal.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\deserialize.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\deserialize.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\extension_type.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\extension_type.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\filesystem.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\filesystem.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\flight.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\flight.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\gdb.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\gdb.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\helpers.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\helpers.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\inference.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\inference.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\init.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\init.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\io.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\io.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\ipc.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\ipc.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\iterators.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\numpy_convert.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\numpy_convert.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\numpy_internal.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\numpy_interop.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\numpy_to_arrow.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\numpy_to_arrow.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\parquet_encryption.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\parquet_encryption.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\pch.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\platform.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\pyarrow.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\pyarrow.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\pyarrow_api.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\pyarrow_lib.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\python_test.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\python_test.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\python_to_arrow.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\python_to_arrow.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\serialize.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\serialize.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\type_traits.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\udf.cc
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\udf.h
Formatting C:\Users\Divyansh\Documents\GitHub\arrow\python\pyarrow\src\arrow\python\visibility.h
Traceback (most recent call last):
  File "C:\Users\Divyansh\Documents\GitHub\arrow\cpp\build-support\run_clang_format.py", line 93, in <module>
    results = lintutils.run_parallel([
              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Divyansh\Documents\GitHub\arrow\cpp\build-support\lintutils.py", line 59, in run_parallel
    procs_batch = [Popen(cmd, **kwargs) for cmd in cmds_batch]
                   ^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Python312\Lib\subprocess.py", line 1538, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [WinError 2] The system cannot find the file specified
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Divyansh\Documents\GitHub\arrow\myenv\Scripts\archery.exe\__main__.py", line 7, in <module>
  File "C:\Users\Divyansh\Documents\GitHub\arrow\myenv\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Divyansh\Documents\GitHub\arrow\myenv\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\Divyansh\Documents\GitHub\arrow\myenv\Lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Divyansh\Documents\GitHub\arrow\myenv\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Divyansh\Documents\GitHub\arrow\myenv\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Divyansh\Documents\GitHub\arrow\myenv\Lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Divyansh\Documents\GitHub\arrow\dev\archery\archery\cli.py", line 299, in lint
    linter(src, fix, iwyu_all=iwyu_all, **checks)
  File "C:\Users\Divyansh\Documents\GitHub\arrow\dev\archery\archery\utils\lint.py", line 464, in linter
    results.extend(python_cpp_linter(src,
  File "C:\Users\Divyansh\Documents\GitHub\arrow\dev\archery\archery\utils\lint.py", line 285, in python_cpp_linter
    yield LintResult.from_cmd(python.run(*args))
                              ^^^^^^^^^^^^^^^^^
  File "C:\Users\Divyansh\Documents\GitHub\arrow\dev\archery\archery\utils\command.py", line 78, in run
    return subprocess.run(invocation, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['python', 'C:\\Users\\Divyansh\\Documents\\GitHub\\arrow\\cpp\\build-support\\run_clang_format.py', '--source_dir', 'C:\\Users\\Divyansh\\Documents\\GitHub\\arrow\\python\\pyarrow\\src\\arrow\\python', '--clang_format_binary', 'clang-format-14', '--fix']' returned non-zero exit status 1.
(myenv) 
Divyansh@LAPTOP-8JH824JM MINGW64 ~/Documents/GitHub
$

@Divyansh200102
Copy link
Contributor Author

@AlenkaF why is this error coming the import import pyarrow.parquet as pq seems to be correct?

=================================== FAILURES ===================================
__________ [doctest] pyarrow._dataset_parquet.ParquetFileWriteOptions __________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> options = pq.ParquetFileWriteOptions(use_dictionary=True, compression="snappy")
UNEXPECTED EXCEPTION: AttributeError("module 'pyarrow.parquet' has no attribute 'ParquetFileWriteOptions'")
Traceback (most recent call last):
  File "/opt/conda/envs/arrow/lib/python3.9/doctest.py", line 1334, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest pyarrow._dataset_parquet.ParquetFileWriteOptions[3]>", line 1, in <module>
AttributeError: module 'pyarrow.parquet' has no attribute 'ParquetFileWriteOptions'
/opt/conda/envs/arrow/lib/python3.9/site-packages/pyarrow/_dataset_parquet.cpython-39-x86_64-linux-gnu.so:None: UnexpectedException

@kou
Copy link
Member

kou commented Feb 25, 2024

It seems that you're using the official Windows Python https://www.python.org/downloads/windows/ and MSYS2.
I think that you can't mix them because it also mixes C++ compiler. The official Windows Python uses Visual Studio and MSYS2 uses MinGW-w64.
Could you use PowerShell instead of MSYS2?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AlenkaF AlenkaF AlenkaF left review comments

Assignees
No one assigned
Labels
awaiting committer review Awaiting committer review Component: Documentation Component: Python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Python][Docs] Document ParquetWriteOptions class
4 participants
@Divyansh200102 @AlenkaF @assignUser @kou