Failing py.test again

[salimfurth]

I've installed GitHub, Anaconda, etc, on a brand new computer and - as occurred in December - I am not passing the test.py tests run at startup. Here's the text output. I can't check this on another computer at the moment because I'm working off-site this week.

C:\Users\Furth\Documents\GitHub\tax-calculator [master ≡]> py.test -m "not requires_pufcsv"
============================= test session starts =============================
platform win32 -- Python 2.7.13, pytest-3.0.5, py-1.4.32, pluggy-0.4.0
rootdir: C:\Users\Furth\Documents\GitHub\tax-calculator, inifile: setup.cfg
collected 297 items

taxcalc\tests\test_behavior.py .......
taxcalc\tests\test_calculate.py ..................
taxcalc\tests\test_consumption.py .......
taxcalc\tests\test_decorators.py ...............F
taxcalc\tests\test_dropq.py ...................F.F..........
taxcalc\tests\test_functions.py ....
taxcalc\tests\test_growth.py ......
taxcalc\tests\test_incometaxio.py ..............
taxcalc\tests\test_macro_elasticity.py .
taxcalc\tests\test_parameters.py .
taxcalc\tests\test_policy.py ..........................
taxcalc\tests\test_records.py .............
taxcalc\tests\test_reforms.py .
taxcalc\tests\test_simpletaxio.py ...................
taxcalc\tests\test_utils.py ........................................F.
taxcalc\tests\filings\forms\test_tax_form.py .............
taxcalc\tests\filings\forms\test_tax_forms.py ..........................................................................

================================== FAILURES ===================================
_____________________________ test_force_no_numba _____________________________

    def test_force_no_numba():
        """
        Force execution of code for non-existence of Numba
        """
        global Magic_calc6

        # Mock the numba module
>       from mock import Mock
E       ImportError: No module named mock

taxcalc\tests\test_decorators.py:296: ImportError
___________________________ test_create_json_table ____________________________

    def test_create_json_table():
        df = DataFrame(data=[[1., 2, 3], [4, 5, 6], [7, 8, 9]],
                       columns=['a', 'b', 'c'])
>       ans = create_json_table(df)

taxcalc\tests\test_dropq.py:342:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
taxcalc\dropq\dropq_utils.py:67: in create_json_table
    row_out.append(format_print(df.loc[idx, col], _type, num_decimals))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

x = 2.0, _type = dtype('int64'), num_decimals = 2

    def format_print(x, _type, num_decimals):
        float_types = [float, np.dtype('f8')]
        frmat_str = "0:.{num}f".format(num=num_decimals)
        frmat_str = "{" + frmat_str + "}"
        try:
            if _type in float_types or _type is None:
                return frmat_str.format(x)
            elif _type == int:
                return str(int(x))
            elif _type == str:
                return str(x)
            else:
>               raise NotImplementedError()
E               NotImplementedError

taxcalc\dropq\dropq_utils.py:21: NotImplementedError
______________________ test_format_print_not_implemented ______________________

    def test_format_print_not_implemented():
        x = np.array([1], dtype='i4')
        with pytest.raises(NotImplementedError):
>           format_print(x[0], x.dtype, 2)
E           Failed: DID NOT RAISE <type 'exceptions.NotImplementedError'>

taxcalc\tests\test_dropq.py:361: Failed
_________________________ test_ascii_output_function __________________________

csvfile = <closed file '<fdopen>', mode 'a' at 0x0000000010281930>
asciifile = <closed file '<fdopen>', mode 'a' at 0x0000000010281C00>

    def test_ascii_output_function(csvfile, asciifile):
        output_test = tempfile.NamedTemporaryFile(mode='a', delete=False)
>       ascii_output(csv_filename=csvfile.name, ascii_filename=output_test.name)

taxcalc\tests\test_utils.py:712:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

csv_filename = 'c:\\users\\furth\\appdata\\local\\temp\\tmpc9idym'
ascii_filename = 'c:\\users\\furth\\appdata\\local\\temp\\tmprdwzm5'

    def ascii_output(csv_filename, ascii_filename):
        """
        Converts csv output from Calculator into ascii output with uniform
        columns and transposes data so columns are rows and rows are columns.
        In an ipython notebook, you can import this function from the utils module.
        """
        # ** List of integers corresponding to the numbers of the rows in the
        #    csv file, only rows in list will be recorded in final output.
        #    If left as [], results in entire file are being converted to ascii.
        #    Put in order from smallest to largest, for example:
        #    recids = [33180, 64023, 68020, 74700, 84723, 98001, 107039, 108820]
        recids = [1, 4, 5]
        # ** Number of characters in each column, must be a nonnegative integer.
        col_size = 15
        # read csv_filename into a Pandas DataFrame
        pdf = pd.read_csv(csv_filename, dtype=object)
        # keep only listed recids if recids list is not empty
        if recids != []:
            def pdf_recid(recid):
                """ Return Pandas DataFrame recid value for specified recid """
                return recid - 1
            recids = map(pdf_recid, recids)  # pylint: disable=bad-builtin
            pdf = pdf.ix[recids]  # pylint: disable=no-member
        # do transposition
        out = pdf.T.reset_index()  # pylint: disable=no-member
        # format data into uniform columns
        fstring = '{:' + str(col_size) + '}'
        out = out.applymap(fstring.format)
        # write ascii output to specified ascii_filename
        out.to_csv(ascii_filename, header=False, index=False,
>                  delim_whitespace=True, sep='\t')
E       TypeError: to_csv() got an unexpected keyword argument 'delim_whitespace'

taxcalc\utils.py:685: TypeError
=========================== pytest-warning summary ============================
WC1 None [pytest] section in setup.cfg files is deprecated, use [tool:pytest] instead.
============================= 3 tests deselected ==============================
=== 4 failed, 290 passed, 3 deselected, 1 pytest-warnings in 68.24 seconds ====

[martinholmer]

@salimfurth, Sorry you're having problems getting started on the new computer.

It looks like you might not have all the Python packages that Tax-Calculator uses. Please review steps 9, 10, and (especially) 11 in this getting started document. After (re?)doing these steps, try the tests again as described in the first code block of 11. Let us know what happens.

[martinholmer]

Further investigation of the py.test errors generated on @salimfurth's new Windows computer (as reported in issue #1172), shows that he has on that computer a version of the Tax-Calculator source code that is over a month out-of-date.

Here is why that seems to be the case. The test_utils.py error he reported was this:

        # write ascii output to specified ascii_filename
        out.to_csv(ascii_filename, header=False, index=False,
>                  delim_whitespace=True, sep='\t')
E       TypeError: to_csv() got an unexpected keyword argument 'delim_whitespace'
taxcalc\utils.py:685: TypeError

But the delim_whitespace=True, text was removed from the source code with the merger of pull request #1092 on December 19, 2016.

This finding would suggest that part of @salimfurth's problem is that he needs to get the latest up-to-date version of the source code on his new Windows computer by taking the four steps (a through d) described in item 1 of the Workflow section of this getting started document.

[salimfurth]

Thanks, @martinholmer - I'm learning something new about the mechanics here. Merging from upstream has knocked out 3 of the 4 errors. The remaining one is that it can't find the module "mock". I'm re-running Step 11 (which did yesterday as well) to see if that fixes it.

[martinholmer]

@salimfurth said:

I'm learning something new about the mechanics here.

Good. Although painful for you, your experiences are helpful to me as I've been asked to review, and hopefully, streamline the getting-started process.

Merging from upstream has knocked out 3 of the 4 errors. The remaining one is that it can't find the module "mock". I'm re-running Step 11 (which did yesterday as well) to see if that fixes it.

Excellent progress! Let me know what happens after Step 11 is redone. It may well be that we have a lingering package installation bug. If so, apologies in advance for you being our Windows tester. (Almost none of the developers are working on Windows.)

[salimfurth]

Thanks, @martinholmer. Happy to blaze the trail to the wilds of Windows. No progress on mock.

Where ought the mock module to be installed? Is it an Anaconda package or a module within Taxcalc?

[martinholmer]

@salimfurth said:

Happy to blaze the trail to the wilds of Windows. No progress on mock.

Where ought the mock module to be installed? Is it an Anaconda package or a module within Taxcalc?

Here is your work-around while we fix the bug that meant the mock package was not installed:

cd C:\Users\Furth\Documents\GitHub\tax-calculator
conda install mock

Then redo the tests and hopefully everything will be OK. Let us know if this work around eliminates all your test failures.

Meanwhile, we'll fix the installation bug that produced your problem.

@MattHJensen @PeterDSteinberg

[MattHJensen]

Meanwhile, we'll fix the installation bug that produced your problem.

@martinholmer, are you referring to the fact that mock is included in environment.yml but not making it into @salimfurth's environment when he activates taxcalc-dev?

[salimfurth]

Thanks again @martinholmer.
I did
conda install mock
and that worked to pass all tests.

[martinholmer]

@salimfurth said:

Thanks again. I did conda install mock and that worked to pass all tests.

Great! I've filed a bug report based on your experience. Thanks for the helpful feedback.

[martinholmer]

Issue #1172 has been resolved for a new Windows user by providing a work around.
But this new user's experience suggest there is a bug in the package installation process as suggested in issue #1173.

[MattHJensen]

@salimfurth, did anything happen when you did activate taxcalc-dev? I don't see taxcalc-dev prepended to your system command prompt, so I'm wondering if that step didn't work.

[salimfurth]

No, I never get any response when I run 'activate taxcalc-dev' - I didn't know I was supposed to.

[MattHJensen]

Got it. Thanks.