Incompatible changes in version 5.1

[perlpunk]

The PyYAML release 5.1 adds some incompatible changes.
If you are getting problems with the new version, that can have several reasons, which are listed here.

Also note that there are things that still need to be fixed. Please wait for release 5.2 and see details at the bottom

The full list of merged PRs can be found here: https://github.com/yaml/pyyaml/blob/master/announcement.msg#L30

The two incompatible changes are:

• Deprecate yaml.load and add FullLoader and UnsafeLoader classes #257 -- Deprecate yaml.load and add FullLoader and UnsafeLoader classes
• Make default_flow_style=False #256 -- Make default_flow_style=False

load: Deprecate yaml.load and add FullLoader and UnsafeLoader classes

For the full story read #257 and #207

YAML is a serialization language, and the yaml.load method allowed to execute arbitrary function calls and import modules, which made it unsafe when loading untrusted YAML.

With this release, it is deprecated to use yaml.load without a Loader argument and will issue a warning:

data = yaml.load("some yaml") # deprecated, warns. It uses FullLoader behind the scenes
data = yaml.load("some yaml", Loader=yaml.SafeLoader) # ok

use safe_load/SafeLoader

yaml.safe_load() always existed for that purpose, and most of you which are getting a warning now should be able to just replace yaml.load with yaml.safe_load:

yaml.safe_load(...)
yaml.load(..., Loader=yaml.SafeLoader)

use full_load/FullLoader

If you are actually using the serialization features (e.g. !!python/object/... tags) and want to load python objects, you should be fine with using the FullLoader:

yaml.load(..., Loader=yaml.FullLoader)
yaml.full_load(...)

FullLoader is pretty safe, but not completely. For an attack, you would need to

• find a module that does something destructive/exploitable in its constructor
• rely on the fact that this module is installed and already loaded

use unsafe_load/UnsafeLoader

There are two things that the FullLoader won't do, though:

• Automatically import the modules for the objects that you want to load
• Execute arbitrary function/method calls

If you need this then you must:

yaml.load(..., Loader=yaml.UnsafeLoader)
yaml.unsafe_load(...)
# or be backwards compatible
yaml.load(..., Loader=yaml.Loader)

yaml.Loader will behave just like yaml.UnsafeLoader

dump: Make default_flow_style=False

#256

YAML has two styles for collections: block and flow style.
The flow style looks a bit similar to JSON or to Python itself:
block key: { flow: mapping, more: items }

To save some space, the default for dumping was to use flow style on lists and dicts at the lowest level of a data structure, that means, that have no child nodes themselves:
yaml.dump(data, default_flow_style=None)

However, many people were using default_flow_style=False to avoid this, so the default didn't seem to be the best.
We decided that we change the default in a major release:
yaml.dump(data, default_flow_style=False)

You can still get the old behaviour by using None instead of False.

Other important changes

• Allow colon in a plain scalar in a flow context #45 -- Allow colon in a plain scalar in a flow context
• Adding support to Unicode characters over codepoint 0xffff #63 -- Adding support to Unicode characters over codepoint 0xffff
• Allow to turn off sorting keys in Dumper #143 -- Allow to turn off sorting keys in Dumper
• Support escaped slash in double quotes "\/" #158 -- Support escaped slash in double quotes "/"

Breakages to be fixed in 5.2

When the default loader yaml.Loader for yaml.load() was changed to FullLoader, it was forgotten that there are two other places where we need to change the default:

• in yaml.add_constructor
• in the class YAMLObject which you can inherit from so that from_yaml and to_yaml are called

Also the logic for detecting special characters was broken, which results in strings with tabs (for example) to be dumped in plain style on python 2.7 on systems with sys.maxunicode <= 0xffff.

PRs are ready:
#273
#274
#276

[perlpunk]

Please see the update. If you are using yaml.add_constructor without specifying a loader, and same for using YAMLObject/from_yaml/to_yaml please wait for release 5.2

[wimglenn]

Where are the docs maintained? Doesn't seem to be a part of this repository directly?

Although the source link was updated for PyYAML-5.1.tar.gz on PyYAMLDocumentation the info remaining about default_flow_style is old (and now, incorrect).

[pombredanne]

I maintain this small library called saneyaml that is a wrapper of PyYAML to get "sane" defaults (mainly safe loading and block style defaults) at https://github.com/nexB/saneyaml
I pushed yesterday a bunch of updates that deal with making this compatible with both PyYAML 3.1x and 5.1x after the report of installation issues from @dankegel in aboutcode-org/saneyaml#1

This may be of help to some to see the changes needed to support both versions:
aboutcode-org/saneyaml@v0.3...v0.4

As a side note, I wished I would not have had the need to write such a wrapper ;)

[dthkao]

Just to summarize the status of PRs:

#273 was replaced with #279
#274 was replaced with #287
#276 was merged

These are all the open tasks in PyYAML release/5.2 branch. The branch also hasn't been touched since March.
@ingydotnet @nitzmahone is there anything else blocking the release?

[mattseymour]

Adding support for TZ would also be greatly appreciated if cutting a release there are currently three pull requests trying to resolve this problem.

• 163
• 113
• 251

[BonnieH]

Do you guys have an approximate release date for 5.2? My code is broken because of these issues. I need to know if I should search for another solution, or if I could wait for the new release. Thank you!

[dthkao]

I've been trying to get someone to confirm that 5.2 can just be cut as is now in #336 ... The branch has been sitting untouched for a while.

[perlpunk]

I was able to talk to @ingydotnet and I'm trying to prepare a 5.2 release now

[perlpunk]

We just released 5.2b1 https://pypi.org/project/PyYAML/5.2b1/

[perlpunk]

We released 5.2: https://pypi.org/project/PyYAML/5.2/