Linguist ignoring part of repository tree

[phetdam]

Hi,

Ran into a strange issue with Linguist today that had me more or less (metaphorically, of course) pulling my hair out until I did a refactoring of the repository tree. Previously I had a repository tree that looked like

.
|-- check
|   |-- check_helpers.h
|   `-- runner.cpp
|-- examples
|   |-- py_gch_example
|   |   |-- <some .py files>
|   |   |-- solvers.c
|   |   `-- tests
|   |       `-- <even more .py files>
|   |-- pkg_version.rst
|   `-- pkg_longdesc.rst
|-- include
|   |-- py_gch.h
|-- pytest.ini
|-- setup.py

I know for sure that the majority of the code in the repo is Python, as the number of pytest unit tests I was writing was exploding. The header files (.h) I expected to be indexed as C++ by default, but the one .c file is ~300 lines, many more lines than my Makefile, so I was quite tilted when after constant refreshing, additional pushing to the repo, even untracking and then retracking of some files, that the .c file didn't show up at all in the language statistics.

After a lot of messing around with .gitattributes, where setting linguist-detectable for examples/py_gch_example didn't do much, I decided to build github-linguist locally and see what was happening. I ended up with this output:

68.93%  C++
15.98%  Makefile
13.47%  Python
1.62%   C

Unfortunately, I can't show the breakdown since I was using less to see the output whenever I passed --breakdown, but please take it on faith that the entire examples directory didn't show up in the output. I thought maybe "examples" was a vendored pattern, but I searched through vendor.yml and the only match for "examples" was for extjs, whose regex doesn't match my repository layout. I did try setting -linguist-vendored for the files in examples/py_gch_examples but that didn't help.

Eventually, I figured that maybe just having "examples" as a directory name was the problem, even though there was nothing in the Linguist README.md to suggest otherwise. So I refactored my directory tree into

.
|-- check
|   |-- check_helpers.h
|   `-- runner.cpp
|-- include
|   |-- py_gch.h
|-- pkg_version.rst
|-- pkg_longdesc.rst
|-- py_gch_demo
|   |-- <.py files from py_gch_example>
|   |--solvers.c
|   `-- tests
|       `-- <.py files from py_gch_example/tests>
|-- pytest.ini
|-- setup.py

Lucky guess. Now github-linguist, when run in my repo root (all changes were committed + pushed), gives me

48.81%  Python
24.21%  C++
21.51%  C
5.48%   Makefile

This is more or less the output I expected. In summary, when I had all my Python and C code in examples/py_gch_example, Linguist failed to detect any of the files within that directory, but when I removed the examples directory and moved the contents of examples/py_gch_example to py_gch_demo, Linguist started giving me the output I expected.

Is this expected behavior? It's not obvious to me that Linguist should simply ignore a directory named "examples", and it's weird that none of the .gitattributes overrides using linguist-detectable, -linguist-vendored, would have no effect. I even got desperate at one point and had * linguist-detectable as my .gitattributes, but Linguist failed to detect the C file even then.

I did "fix" the problem on my end but I don't think this is behavior one would expect from Linguist. Since I'm hesitant to open an issue claiming this is a bug since it might very well be me just acting like a monkey (not the first time :p), I decided to open a discussion.

Hoping that someone's insights might shed some light on what happened. Thanks!

[lildude]

Linguist failed to detect any of the files within that directory, but when I removed the examples directory and moved the contents of examples/py_gch_example to py_gch_demo, Linguist started giving me the output I expected.

Is this expected behavior? It's not obvious to me that Linguist should simply ignore a directory named "examples", and it's weird that none of the .gitattributes overrides using linguist-detectable, -linguist-vendored, would have no effect. I even got desperate at one point and had * linguist-detectable as my .gitattributes, but Linguist failed to detect the C file even then.

Yes, this is expected behaviour and you missed the one other type that is ignored by default, and examples falls into it: documentation... thanks to this:

https://github.com/github/linguist/blob/299318747a903a90581f46e637f7d3be387ff844/lib/linguist/documentation.yml#L17

You'd need to implement the documentation override to force the files in examples to be considered.

[phetdam]

Hm, looks like I was looking in the wrong places. For some I never considered the possibility that "examples" might be related to documentation.

Thanks!