convergence and large stochastic error

[kw600]

I have asked about the large stochastic error before. It seems like the current release version has not fix the issue of very large stochastic error. I understand I can set meaningful_factor to minimize the FC gradient. In the same time, for very large supercells (>300 atoms), is it possible that I optimize the structure in the first few populations, and when it is somehow small, I turn off the strcutural minimization and only allow minimization of FC gradient. Please look at the following output:

Free energy = -279526.10257191 +-       0.35424688 meV
FC gradient modulus =      68.43811147 +-  966196.17323290 bohr^2
Struct gradient modulus =       0.00000000 +-       0.00000000 meV/A
Kong-Liu effective sample size =  1481.329908637985

Total force on the centroids [eV/A]:
   0)      -0.000601      0.004419      0.000918  +-      -0.000009      0.000019      0.000226
   1)      -0.003527     -0.002730      0.000918  +-      -0.000012     -0.000017      0.000226
   2)       0.004127     -0.001689      0.000918  +-       0.000021     -0.000002      0.000226
   3)      -0.000000     -0.000000     -0.002809  +-       0.000000      0.000000      0.000274
   4)       0.000000      0.000000     -0.003807  +-       0.000000      0.000000      0.000249
   5)      -0.003582     -0.001491      0.000508  +-       0.000014      0.000070      0.000219
   6)       0.000499      0.003847      0.000508  +-       0.000054     -0.000047      0.000219
   7)       0.003082     -0.002356      0.000508  +-      -0.000067     -0.000023      0.000219
   8)       0.000000      0.000000      0.000299  +-       0.000000      0.000000      0.000244
   9)       0.000000      0.000000      0.003246  +-       0.000000      0.000000      0.000274
  10)       0.003492      0.004311     -0.010705  +-      -0.000063     -0.000148      0.001054
  11)       0.001987     -0.005180     -0.010705  +-      -0.000097      0.000128      0.001054
  12)      -0.005479      0.000869     -0.010705  +-       0.000159      0.000019      0.001054
  13)      -0.001462     -0.001113      0.000231  +-       0.000264      0.000064      0.000309
  14)       0.001696     -0.000710      0.000231  +-      -0.000187      0.000196      0.000309
  15)      -0.000233      0.001823      0.000231  +-      -0.000077     -0.000260      0.000309
  16)       0.008063      0.005690     -0.001177  +-      -0.000005     -0.000370      0.000304
  17)       0.000896     -0.009828     -0.001177  +-      -0.000318      0.000189      0.000304
  18)      -0.008960      0.004138     -0.001177  +-       0.000323      0.000181      0.000304
  19)       0.003011      0.003096      0.008346  +-      -0.000059     -0.000134      0.001071
  20)       0.001176     -0.004156      0.008346  +-      -0.000087      0.000118      0.001071
  21)      -0.004187      0.001060      0.008346  +-       0.000146      0.000016      0.001071
  22)       0.000278      0.005223     -0.006646  +-      -0.000159      0.000089      0.001068
  23)      -0.004663     -0.002371     -0.006646  +-       0.000002     -0.000183      0.001068
  24)       0.004384     -0.002853     -0.006646  +-       0.000157      0.000093      0.001068
  25)       0.000000      0.000000      0.001185  +-      -0.000000     -0.000000      0.000166
  26)      -0.000126      0.004246      0.001719  +-      -0.000386      0.000101      0.000334
  27)      -0.003614     -0.002232      0.001719  +-       0.000105     -0.000385      0.000334
  28)       0.003740     -0.002014      0.001719  +-       0.000280      0.000283      0.000334
  29)       0.005199      0.000217     -0.002414  +-       0.000295      0.000181      0.000377
  30)      -0.002411     -0.004611     -0.002414  +-       0.000009     -0.000346      0.000377
  31)      -0.002787      0.004394     -0.002414  +-      -0.000304      0.000165      0.000377
  32)      -0.000055      0.005949      0.010564  +-      -0.000179      0.000110      0.001054
  33)      -0.005125     -0.003022      0.010564  +-      -0.000005     -0.000210      0.001054
  34)       0.005180     -0.002927      0.010564  +-       0.000185      0.000100      0.001054
  35)      -0.005012      0.000205      0.002085  +-      -0.000068      0.000032      0.001437
  36)       0.002683      0.004238      0.002085  +-       0.000062      0.000043      0.001437
  37)       0.002328     -0.004443      0.002085  +-       0.000006     -0.000075      0.001437
  38)       0.005422      0.004969     -0.002801  +-      -0.000039      0.000013      0.001406
  39)      -0.007015      0.002211     -0.002801  +-       0.000009     -0.000040      0.001406
  40)       0.001593     -0.007180     -0.002801  +-       0.000031      0.000028      0.001406

The final force on the structure is less than 0.01 ev/A and I only allow minimization of FC gradient. Is it safe to say it is converged? I do not want to turn on the structural minimization since the primitive cell is very large and there are so many degreees of freedom on forces, which makes the convergence speed so slow.

Another question is about the bubble correction (or it is called static correction?). Do you have any idea why the SSCHA matrix looks correct (positive definite and nice) but the SSCHA + bubble correction matrix is very bad (lots of imaginary modes even at Gamma points). Somehow I feel like it is due to the SSCHA matrix is not well converged and in the same time I need to add more configurations into the last population to converge the bubble correction? In the same time I am also afraid it is due to I turn off the structural minimization. Can you give me any idea?

By the way, I am using ensemble.get_free_energy_hessian to get the SSCHA + bubble correction and I also see the Spectral.get_static_correction_interpolated can do the same thing. Do these two functions give consistent results or I should use one rather than the other?

This is a quite long question. Many thanks for any help.

Kang

[mesonepigreco]

Hi, the error fix is currently in the fourier_gradient branch and will be merged in version 1.4, to be released.

is it possible that I optimize the structure in the first few populations, and when it is somehow small, I turn off the strcutural minimization and only allow minimization of FC gradient?

Yes, this is a good approach to speed up the calculation.

The final force on the structure is less than 0.01 ev/A and I only allow minimization of FC gradient. Is it safe to say it is converged? I do not want to turn on the structural minimization since the primitive cell is very large and there are so many degreees of freedom on forces, which makes the convergence speed so slow.

The forces' values seem small; however, it depends a bit on the system if it is enough. Anyway, the larger the supercell, the more negligible the number of degrees of freedom of the structure with respect to the auxiliary FC is, so the complete minimization should be mostly unaffected by the minimization of the structure. Have you tested with the same ensemble to run the minimization of the structure?
It may be useful to plot the behavior of frequencies, free energy, and gradients as a function of the minimization to understand if it is converging. This is achieved with

$ sscha-plot-data.py minim

If you set up the saving of the frequencies as explained in the tutorials: (for example, here )

Another question is about the bubble correction (or it is called static correction?). Do you have any idea why the SSCHA matrix looks correct (positive definite and nice) but the SSCHA + bubble correction matrix is very bad (lots of imaginary modes even at Gamma points). Somehow I feel like it is due to the SSCHA matrix is not well converged and in the same time I need to add more configurations into the last population to converge the bubble correction? In the same time I am also afraid it is due to I turn off the structural minimization. Can you give me any idea?

This depends on the system. The Free energy hessian in the bubble approximation requires more configurations than the SSCHA minimization; therefore, it may be harder to converge.
You can try to check if the results change with the number of configurations employed in your ensemble by splitting the ensemble in two and comparing the hessian computed with half the configurations and the total number of configurations.

For example, see this FAQ for a more detailed explanation of how to do it.

By the way, I am using ensemble.get_free_energy_hessian to get the SSCHA + bubble correction and I also see the Spectral.get_static_correction_interpolated can do the same thing. Do these two functions give consistent results or I should use one rather than the other?

The Free energy hessian computes the Hessian only in the q grid commensurate with the supercell. It can use the bubble approximation (if you set include_v4=False) or even the full hessian beyond the bubble approximation (if include_v4=True).
get_static_correction_interpolated performs the same calculation but interpolates the q-mesh during the calculation, allowing to simulate an effectively bigger supercell; however, only in the bubble approximation. They are called "static", representing the phonon energy only at zero frequency.
For more information on their difference, there are these two tutorials:
Free energy hessian
Static Correction with Interpolation
Another tutorial on the static correction with Interpolation

Hope this helps,
Lorenzo

[kw600]

Dear Lorenzo,

Many thanks for your response. It is really helpful. I have read the mentioned 3 tutorials before but now you explanation makes me more clear.

I tried to use 512 configurations and then added 512 more confugurations (use ensemble.merge and 1024 totally) to calculate the bubble correction. I saved the odd correction matrix and it really changes a lot. Although both give many imaginary modes at gamma point, the 1024 one gives much smaller imaginary values. I checked the odd correction matrix and the 1024 one has smaller matrix elements (50% of the 512 one). I will add more configurations to see if the bubble is converged.

Best wishes,
Kang