Are Tax-Calculator results too sensitive to substitution effect elasticity?

[rickecon]

I ran some tests today using TCJA_Senate.json as my reform and with the following behavioral assumptions json:

{
  "behavior": {
        "_BE_sub": {"2018": [0.25]},
        "_BE_cg": {"2018": [-3.45]}
    },
  "consumption": {
    },
  "growdiff_response": { 
  "_ACPIU": {"2017": [-0.0025]}
  },
  "growdiff_baseline": {}
}

The difference in government revenue over the 10-year budget window for that simulation is -$530 billion. I ran some other simulations of the TCJA_Senate.json reform in which I varied the substitution effect elasticity (_BE_sub) and the capital gains elasticity (_BE_cg). These are the results I got.

description	_BE_sub	_BE_cg	growdiff_response	10-yr rev. chg.
Original	0.250	-3.450	-0.0025	-$530 B
No behavior	0.000	0.000	0.0000	-$1,300 B
_BE_cg down	0.250	- 1.725	-0.0025	-$556 B
_BE_sub down	0.125	- 3.450	-0.0025	-$906 B
_BE_sub up	0.400	- 3.450	-0.0025	-$76 B

Going from an elasticity of substitution of 0.4 (which is still within a reasonable range from the literature) to a lower elasticity of 0.125 wipes out nearly $1 trillion of lost revenue. That seems like too much sensitivity.

@jdebacker pointed out one possible culprit. The effect of the elasticity on behavior is the following. Let income be Inc. Let MTR_b be the marginal tax rate in the baseline scenario, and let MTR_r be the marginal tax rate in the reform scenario. The behavioral effect of a change in MTR_r is approximated as the percent change in 1 - MTR multiplied by the substitution effect elasticity of taxable income _BE_sub times income Inc.

change in income = [(1 - MTR_r) / (1 - MTR_b) - 1] x _BE_sub x Inc

This is shown in lines 157-168 of behavior.py. The key value is the percent change in 1 - MTR from line 165 of behavior.py:

pch = (1 - MTR_r) / (1 - MTR_b) - 1

If I describe the values of pch for the Senate reform, I get 30 filers having a value greater than 0.99 (0.0139%) and 48 filers having a value less than -0.3 (0.0223%). These observations with really large values are what are making Tax-Calculator so sensitive to the behavioral assumption of the elasticity of substitution of taxable income (_BE_sub). Is this right?

ANALYSIS OF pch

Stat	Value
max	5.040
mean	0.050
min	-0.991

Criterion	Count	Percent
obs > 0.30	11,236	5.200%
obs > 0.99	30	0.014%
obs < -0.30	48	0.022%

ANALYSIS OF MTR_b (denominator)

Stat	Value
max	1.197
mean	0.302
min	-0.415

Criterion	Count	Percent
obs > 0.80	10	0.005%
obs > 0.99	3	0.001%
obs < -0.30	2,626	1.218%
0.98 < obs < 1.02	1	0.004%

I suggest that we truncate either the MTR's used in the behavioral calculation or we truncate the pch values. In fact, I don't think that the pch expression is valid for MTR's >= 1.0.
@MattHJensen @martinholmer @jdebacker @hayleeefay

[rickecon]

@martinholmer. I am OK with the title of this issue being changed to a question. But it is not a question in my mind. My comment above is strong evidence that the results are too sensitive to the substitution effect elasticity. If I can generate a loss of only $76 billion over 10 years from the Senate reform by using an elasticity of 0.4 (the most common value for this parameter from the literature) versus a 10-year loss of $530 billion with an elasticity value of 0.25, this is too sensitive. Furthermore, I document that the sensitivity is coming from (MTR_b > 1.0 and MTR_r > 1.0).

[MattHJensen]

I see two potentially useful followups here:

• Check to see how the percent change in overall revenue going from ETI = 0 to ETI = 0.4 differs when you run the analysis with and without the records with extreme mtrs (under some defn of extreme)
• Identify whether the high and low MTRs are reasonable or not by examining the records. I'm not sure a priori whether 119% MTR or -41% MTR are reasonable or not.

cc @feenberg on #1668.

[martinholmer]

@MattHJensen said in issue #1668:

I see two potentially useful followups here:

• Check to see how the percent change in overall revenue going from ETI = 0 to ETI = 0.4 differs when you run the analysis with and without the records with extreme mtrs (under some defn of extreme).
• Identify whether the high and low MTRs are reasonable or not by examining the records. I'm not sure a priori whether 119% MTR or -41% MTR are reasonable or not.

Beginning this morning, I'm researching these issues. I'll be reporting my findings as the day progresses.

@rickecon

[martinholmer]

@rickecon raised a number of important issues in #1668. The Tax-Calculator developers have been very busy recently, so we haven't paid as much attention to these issues as we would normally have. My work on #1668 has started by trying to reproduce the kind of behavior response results for the TCJA_Senate.json reform that @rickecon describes in #1668. In doing this, I'm following his discussion by focusing only on the behavioral substitution response to the TCJA_Senate.json reform.

I show first my initial replication results and then after that the Python script that generates those results.

RESULTS

$ cd Tax-Calculator
$ ls -l puf.csv
-rw-r--r--  1 mrh  staff  53743252 Nov 22 09:41 puf.csv
$ python issue1668.py
iMac:Tax-Calculator mrh$ python issue1668.py
2018_CURRENT_LAW    itax_rev($B)= 1795.8
2018_CURRENT_LAW    ptax_rev($B)= 1141.8
2018_CURRENT_LAW    comb_rev($B)= 2937.7
CURRENT-LAW MTR HISTOGRAM WHEN wrt_full_compensation=True:
[     0 215523      2]
[ -9.00000000e+99  -1.00000000e+00   1.00000000e+00   9.00000000e+99]
max MTR value = 119.7%
2018_REFORM_STATIC  itax_rev($B)= 1669.7
2018_REFORM_STATIC  ptax_rev($B)= 1141.8
2018_REFORM_STATIC  comb_rev($B)= 2811.5
REFORM MTR HISTOGRAM WHEN wrt_full_compensation=True:
[     0 215523      2]
[ -9.00000000e+99  -1.00000000e+00   1.00000000e+00   9.00000000e+99]
max MTR value = 115.0%
{2018: {u'_BE_sub': [0.4]}}
2018_REFORM_DYNAMIC itax_rev($B)= 1746.9
2018_REFORM_DYNAMIC ptax_rev($B)= 1152.6
2018_REFORM_DYNAMIC comb_rev($B)= 2899.5

Summarizing the results, we have:

                                     absolute($B)   relative
STATIC  2018 rev effect: 2811.5 - 2937.7 = -126.2   100.0
DYNAMIC 2918 rev effect: 2899.5 - 2937.7 =  -38.2    30.3

In other words, the 0.4 behavioral substitution effect, raises incomes enough to eliminate almost 70% of the static revenue loss of the reform.

SCRIPT (named issue1668.py)

from __future__ import print_function
from taxcalc import *
import numpy as np

reform_filename = './taxcalc/reforms/TCJA_Senate.json'
assump_filename = './issue1668assump.json'


def show_mtr_histogram(calc, title):
    print(title + ' WHEN wrt_full_compensation=True:')
    mtr_ptax, mtr_itax, mtr_comb = calc.mtr(wrt_full_compensation=True)
    hist = np.histogram(mtr_comb, bins=[-9e99,-1.0,1.0,9e99])
    print(hist[0])
    print(hist[1])
    print('max MTR value = {:.1f}%'.format(100 * mtr_comb.max()))


# use proprietary PUF input file
recs = Records()

# specify Calculator object for current-law policy
pol = Policy()
calc1 = Calculator(policy=pol, records=recs, verbose=False)

# calculate aggregate current-law tax liabilities for 2018
calc1.advance_to_year(2018)
calc1.calc_all()
itax_rev1 = calc1.weighted_total('iitax')
ptax_rev1 = calc1.weighted_total('payrolltax')
comb_rev1 = itax_rev1 + ptax_rev1

# read JSON reform file and use static analysis assumptions
params = Calculator.read_json_param_objects(reform=reform_filename,
                                            assump=assump_filename)

# specify Calculator object for static analysis of reform policy
pol.implement_reform(params['policy'])
if pol.reform_errors:  # check for reform error messages
    print(pol.reform_errors)
    exit(1)
calc2 = Calculator(policy=pol, records=recs, verbose=False)

# calculate reform tax liabilities for 2018 using static analysis assumptions
calc2.advance_to_year(2018)
calc2.calc_all()
itax_rev2 = calc2.weighted_total('iitax')
ptax_rev2 = calc2.weighted_total('payrolltax')
comb_rev2 = itax_rev2 + ptax_rev2

# specify Calculator object for dynamic analysis of reform policy
behv = Behavior()
behv.update_behavior(params['behavior'])
calc3b = Calculator(policy=pol, records=recs, behavior=behv, verbose=False)

# calculate reform tax liabilities for 2018 using dynamic analysis assumptions
calc3b.advance_to_year(2018)
calc3 = Behavior.response(calc1, calc3b)
itax_rev3 = calc3.weighted_total('iitax')
ptax_rev3 = calc3.weighted_total('payrolltax')
comb_rev3 = itax_rev3 + ptax_rev3

# print results for 2018
print('2018_CURRENT_LAW    itax_rev($B)= {:.1f}'.format(itax_rev1 * 1e-9))
print('2018_CURRENT_LAW    ptax_rev($B)= {:.1f}'.format(ptax_rev1 * 1e-9))
print('2018_CURRENT_LAW    comb_rev($B)= {:.1f}'.format(comb_rev1 * 1e-9))
show_mtr_histogram(calc1, 'CURRENT-LAW MTR HISTOGRAM')
print('2018_REFORM_STATIC  itax_rev($B)= {:.1f}'.format(itax_rev2 * 1e-9))
print('2018_REFORM_STATIC  ptax_rev($B)= {:.1f}'.format(ptax_rev2 * 1e-9))
print('2018_REFORM_STATIC  comb_rev($B)= {:.1f}'.format(comb_rev2 * 1e-9))
show_mtr_histogram(calc2, 'REFORM MTR HISTOGRAM')
print(params['behavior'])
print('2018_REFORM_DYNAMIC itax_rev($B)= {:.1f}'.format(itax_rev3 * 1e-9))
print('2018_REFORM_DYNAMIC ptax_rev($B)= {:.1f}'.format(ptax_rev3 * 1e-9))
print('2018_REFORM_DYNAMIC comb_rev($B)= {:.1f}'.format(comb_rev3 * 1e-9))

@MattHJensen @feenberg

[martinholmer]

When re-running the script described at the bottom of this comment after making the #1698 changes to the handling of MTRs at or above 100% in the Behavior.response function, the script results are exactly the same. This is not surprising because the revenue results from the script are rounded to the nearest one-tenth of a billion dollars and there were only two filing units (out of a total of 215,525 filing units in puf.csv) that have MTRs at or above 100% in 2018.

So, we are still left with the result that assuming _BE_sub equal to 0.4 produces an increase in income in a dynamic analysis of the TCJA_Senate.json reform that is large enough to generate income and payroll taxes that offset roughly 70% of the static revenue loss. Assuming _BE_sub equal to 0.2 produces a response that offset roughly 35% of the static revenue loss. And, of course, assuming _BE_sub equal to 0.0 produces no response and so none of static revenue loss is offset by behavioral response.

Maybe there is something else wrong with the Behavior.response function logic. I don't know much about the research literature and the basic logic of the Behavior.response function was specified before I started working on Tax-Calculator. So, others are better equipped to investigate that option. But the mishandling of MTRs at or above 100% (which is fixed in #1698) is not the cause of results described in this issue.

@rickecon @jdebacker @MattHJensen @feenberg

[martinholmer]

@MattHJensen said in issue #1668:

Identify whether the high and low MTRs are reasonable or not by examining the records. I'm not sure a priori whether 119% MTR or -41% MTR are reasonable or not.

We've had long conversations about high MTRs. For example, on January 26, 2016, @MattHJensen opened issue #555 (entitled "MTRs respond oddly to double of standard deduction - high priority" with this statement:

I believe there is a bug in our marginal tax rate function. I am comparing baseline to a reform scenario where the standard deduction is doubled; for some taxpayers, marginal tax rates wrt e00200p increase under the reform by as much as 39%. This is quite unexpected.

After fifty comments, @MattHJensen removed the Bug label and closed the issue on February 3, 2016. And then @GoFroggyRun wrote a description of some of these "odd" cases.

Likewise for negative MTRs. For example, just two weeks ago in the discussion of #1645, @codykallen pointed out the reason some filing units have substantially negative MTRs:

By the way, under all four of the scenarios described above, the lowest MTR on e00900 is always the same (-0.6). This is the MTR for individuals who have negative tax liability due to the EITC and the ACTC, since the EITC has a phase-in rate of 45% and the ACTC has a phase-in rate of 15%, resulting in the -60% MTR.

Note that @codykallen was referring to MTRs computed with wrt_full_compensation set to False, while the Behavior.response function sets that parameter equal to True.

All this means that even though the Calculator.mtr function or the basic tax logic of Tax-Calculator may still have bugs, the issue of "extreme" MTRs has been investigated extensively.

And, there may be a flaw in the implmentation of the Behavior.response function.

Or, just maybe, the massive reduction in MTRs for high-income filers in the TCJA reform, combined with high response parameters (like _BE_sub equal to 0.4), simply lead to large increases in income, and hence, large offsets to the reform's static revenue loss.

@feenberg @rickecon @jdebacker @Amy-Xu @andersonfrailey @hdoupe

[rickecon]

My thought is that the expression for the change in income is at best a first-order approximation (high approximation error) for the true delta.

change in income = [(1 - MTR_r) / (1 - MTR_b) - 1] x _BE_sub x Inc_0

and %chg(I) = %chg(1 - MTR) x _BE_sub

This expression comes from a simple static Euler equation for labor supply, which has a distortionary term (1 - MTR) in it [see equation (1) of Saez, Slemrod Giertz, 2012].

Try this thought experiment. Suppose that MTR_r=0.2, _BE_sub=0.2, and Inc_0=100 are fixed. Look at the change in income for values of `MTR_b=0.98, 0.99, 0.999, 1.00, 1.001, 1.01, and 1.02.

MTR_b	MTR_r	_BE_sub	Inc_0	change in income
0.980	0.2	0.2	100	+800
0.990	0.2	0.2	100	+1,600
0.999	0.2	0.2	100	+16,000
1.000	0.2	0.2	100	+Inf or -Inf
1.001	0.2	0.2	100	-16,000
1.010	0.2	0.2	100	-1,600
1.020	0.2	0.2	100	-800

Notice how there is an asymptote at MTR_b=1, and that the effect on income can be either positive infinity or negative infinity at that asymptote depending on from which direction you are coming. This calculation for the elasticity of taxable income is not appropriate when MTR's can be greater than one.
@martinholmer @MattHJensen @jdebacker @feenberg

[martinholmer]

@rickecon, What is the implication of your recent comment for the Behavior.response() code?

Can you make a suggestion about how you think the code should be changed now that the code handles very high MTRs (at or above 100%) better than before #1698?

[martinholmer]

Pull request #1700 adds optional tracing logic to the Behavior.response method. That new capability can be used to add some information to the discussion of issue #1668.

The results shown below are generated by the same script and the same reform/assumption files as described earlier in this conversation. The only change in the script to that the call to Behavior.response now has a trace=True argument. The basic results are unchanged; here are the tracing results:

*** TRACE for variable pch
*** Histogram:
[    0     2   392  5113  8977 72597 90066 24962 12779   609    28]
[ -9.00000000e+99  -1.00000000e+00  -5.00000000e-01  -2.00000000e-01
  -1.00000000e-01  -1.00000000e-05   1.00000000e-05   1.00000000e-01
   2.00000000e-01   5.00000000e-01   1.00000000e+00   9.00000000e+99]
*** Person-weighted mean= 0.03
*** Dollar-weighted mean= 0.14
*** TRACE for variable sub
*** Histogram:
[ 9485  4184 74471 60805 32486 22837  9941  1316]
[ -9.00000000e+99  -1.00000000e+03  -1.00000000e-01   1.00000000e-01
   1.00000000e+03   1.00000000e+04   1.00000000e+05   1.00000000e+06
   9.00000000e+99]
*** Person-weighted mean= 1414.93

So, the TCJA_Senate.json reform leaves the MTR on taxpayer earnings unchanged (that is, pch is essentially zero) for 72,597 filing units and raises the MTR (that is, pch is negative) for 14,484 filing units. The remaining 128,444 filing units experience a reduction in MTR (that is, pch is positive). And some of those experience very large MTR reductions: 28 filing units see their MTR drop so much that pch is greater than one (that is, the after-tax marginal rate doubles or more) and another 609 see their MTR drop so that pch is between 0.5 and 1.0 (where 1.0 is doubling).

Given that those with large positive pch values are typically high-income, the 0.4 substitution elasticity generates large increases in taxable income. At the extreme end of the sub distribution, we have 1,316 filing units who are simulated to have an increase in taxable income of one million dollars or more. And there are another 9,941 filing units who are simulated to have an increase in taxable income of between $100,000 and $1,000,000.

It seems to me the questions @rickecon and @jdebacker posed are: "Are these results reasonable? And, if not, what is the matter with Tax-Calculator?" If you think these results are unreasonable, you need to demonstrate one or more of the following things:

• The changes in MTR under the TCJA_Senate.json are incorrect because either that reform is not specified quite right or the code that implements that reform is not quite correct.
• The logic of the Behavior.response function is flawed.

@MattHJensen @feenberg @Amy-Xu

[martinholmer]

@rickecon and @jdebacker, Have either of you done any more investigation of the issue #1668 that you raised about a month ago? In a private discussion of this issue, @MattHJensen said, given a 0.40 substitution elasticity and the nature of the TCJA reform, he wasn't too surprised about the results you got. Are your planning any further investigation into the matter? If not, shall we close issue #1668? Or is there benefit to keeping it open?

[martinholmer]

Issue #1668 has been resolve by the merge of pull request #1858.