Decrease memory usage #83
Milestone
Comments
|
Why don't you have major categories of mime types stored in separate ruby files and allow users to load only specific ones into the global types array as required? |
Merged
jsonn
pushed a commit
to jsonn/pkgsrc
that referenced
this issue
Jun 8, 2015
== 2.6.1 / 2015-05-25
* Bugs:
* Make columnar store handle all supported extensions, not just the first.
* Avoid circular require when using the columnar store.
== 2.6 / 2015-05-25
* New Feature:
* Columnar data storage for the MIME::Types registry, contributed by Jeremy
Evans (@jeremyevans). Reduces default memory use substantially (the mail
gem drops from 19 Mib to about 3 Mib). Resolves
{#96}[mime-types/ruby-mime-types#96],
{#94}[mime-types/ruby-mime-types#94],
{#83}[mime-types/ruby-mime-types#83]. Partially
addresses {#64}[mime-types/ruby-mime-types#64]
and {#62}[mime-types/ruby-mime-types#62].
* Development:
* Removed caching of deprecation messages in preparation for mime-types 3.0.
Now, deprecated methods will always warn their deprecation instead of only
warning once.
* Added a logger for deprecation messages.
* Renamed <tt>lib/mime.rb</tt> to <tt>lib/mime/deprecations.rb</tt> to not
conflict with the {mime}[https://rubygems.org/gems/mime] gem on behalf of
the maintainers of the {Praxis Framework}[http://praxis-framework.io/].
Provided by Josep M. Blanquer (@blanquer),
{#100}[mime-types/ruby-mime-types#100].
* Added the columnar data conversion tool, also provided by Jeremy Evans.
* Documentation:
* Improved documentation and ensured that all deprecated methods are marked
as such in the documentation.
* Development:
* Added more Ruby variants to Travis CI.
* Silenced deprecation messages for internal tools. Noisy deprecations are
noisy, but that's the point.
== 2.5 / 2015-04-25
* Bugs:
* David Genord (@albus522) fixed a bug in loading MIME::types cache where a
container loaded from cache did not have the expected +default_proc+,
{#86}[mime-types/ruby-mime-types#86].
* Richard Schneeman (@schneems) provided a patch that substantially reduces
unnecessary allocations.
* Documentation:
* Tibor Szolár (@flexik) fixed a typo in the README,
{#82}[mime-types/ruby-mime-types#82]
* Fixed {#80}[mime-types/ruby-mime-types#80],
clarifying the relationship of MIME::Type#content_type and
MIME::Type#simplified, with Ken Ip (@kenips).
* Development:
* Juanito Fatas (@JuanitoFatas) enabled container mode on Travis CI,
{#87}[mime-types/ruby-mime-types#87].
* Moved development to a mime-types organization under
{mime-types/ruby-mime-types}[https://github.com/mime-types/ruby-mime-types].
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
As discussed at RubyConf, the mail gem uses only a small part of the information provided by mime-types, but has to pay the memory penalty for loading mime-types, which is over twice as much memory as that taken by the rest of the mail library.
As mail only needs a small part of the information that mime-types provides, I wrote a patch to mail that extracts only the information from mime-types that mail uses, resulting in a 2/3 reduction in the amount of memory used (see mikel/mail#829). But this has compatibility issues because it can't support the same API that mime-types offers to add types at runtime (
Mime::Type.register). So it's either going to require a major version bump for mail, and dropping the dependency on mime-types, or finding some way to reduce the amount of memory used by mime-types.We discussed various ways to reduce the amount of memory that mime-types uses, such as only loading parts that are needed without loading the entire data set, or storing the information in a file-based database.
Anyway, I'm opening this ticket so we can discuss possible ideas/implementations for reducing memory usage in mime-types.
The text was updated successfully, but these errors were encountered: