Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add class to handle value filling #46

Draft
wants to merge 77 commits into
base: main
Choose a base branch
from
Draft

Add class to handle value filling #46

wants to merge 77 commits into from

Commits on Jun 6, 2024

  1. Define a first draft for a Normalizer "meta-class"

    louisPoulain committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    a766beb View commit details
    Browse the repository at this point in the history

  2. Extend normaliser class with new method. 

    Modify Standardizer to ensure compatibility with the new parent class and method
    louisPoulain committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    ddadee9 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. Continue working on normalizer class. Experimental, still need to be … 

    …tested with the original standardizer class
    louisPoulain committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    6bf868c View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2024

  1. Changed a lot the meta-normalizer architecture.

    louisPoulain committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    a0d030c View commit details
    Browse the repository at this point in the history

  2. Early testing with the meta class and the initial standardizer class.

    louisPoulain committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    3067f28 View commit details
    Browse the repository at this point in the history

  3. Update gitignore to hide test files

    louisPoulain committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    c261abc View commit details
    Browse the repository at this point in the history

  4. Add MinMaxScaler

    louisPoulain committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    1a7764b View commit details
    Browse the repository at this point in the history

  5. Add and perform tests for MinMaxScaler

    louisPoulain committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    d52f86a View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Change the testing to a more general framework.

    louisPoulain committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    5b71fd7 View commit details
    Browse the repository at this point in the history

  2. Correct a few issues

    louisPoulain committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    e9fa689 View commit details
    Browse the repository at this point in the history

  3. Add and test MinMaxScaler

    louisPoulain committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    b4f5333 View commit details
    Browse the repository at this point in the history

  4. Add RobustScaling (standard norma with median and iqr)

    louisPoulain committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    181f8b8 View commit details
    Browse the repository at this point in the history

  5. Make the test file work with an unlimited number of nromalizer subcla… 

    …sses
    louisPoulain committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    ca556fe View commit details
    Browse the repository at this point in the history

  6. Add MaxAbsScaler, BoxCox and YeoJohnson transformations

    louisPoulain committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    8c2761f View commit details
    Browse the repository at this point in the history

  7. Changed minor things

    louisPoulain committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    5f562c6 View commit details
    Browse the repository at this point in the history

  8. Minor changes, this test script is vowed to disappear

    louisPoulain committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    7f3bd13 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Modification of the naming

    louisPoulain committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    efb00d8 View commit details
    Browse the repository at this point in the history

  2. Modofication to take into account the new class

    louisPoulain committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    e4a91f1 View commit details
    Browse the repository at this point in the history

  3. Modif ignore

    louisPoulain committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    869472f View commit details
    Browse the repository at this point in the history

  4. Add a try/except to test if normalizer was fitted. To be changed in t… 

    …he future
    louisPoulain committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    c6b0245 View commit details
    Browse the repository at this point in the history

  5. Rename some parameters and change a bit the structure. Remove RobustS… 

    …caler (issues with computing the median)
    louisPoulain committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    6e4b25a View commit details
    Browse the repository at this point in the history

  6. Add function to bypass dask.nanmedian (not working currently)

    louisPoulain committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    68c9ae2 View commit details
    Browse the repository at this point in the history

  7. Few text changes

    louisPoulain committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    65c88d2 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2024

  1. Add fillvalue in arguments, modify the way an object is created from … 

    …to_dict
    louisPoulain committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    e30ed07 View commit details
    Browse the repository at this point in the history

  2. Reduce tolerance to 1e-7 for allclose tests

    louisPoulain committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    bcbe64d View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. Minor change

    louisPoulain committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    86f8e64 View commit details
    Browse the repository at this point in the history

  2. Add logger message to track the multinormalizer.

    louisPoulain committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    f25ea28 View commit details
    Browse the repository at this point in the history

  3. Debug a bit the lengths of datasets

    louisPoulain committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    61399a2 View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2024

  1. Add default normalizer

    louisPoulain committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    41b7240 View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Move save and load to the super-class

    louisPoulain committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    e79aec4 View commit details
    Browse the repository at this point in the history

  2. Remove old tests for multinormalizer. Reimplement them as pytest tests

    louisPoulain committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    8ee56b7 View commit details
    Browse the repository at this point in the history

  3. Add method to gather class attributes

    louisPoulain committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    6416f74 View commit details
    Browse the repository at this point in the history

  4. Correct typo in confest

    louisPoulain committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    90f1c57 View commit details
    Browse the repository at this point in the history

  5. Correct issue in conftest

    louisPoulain committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    3968210 View commit details
    Browse the repository at this point in the history

  6. Remove "return" statement

    louisPoulain committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    0d3171b View commit details
    Browse the repository at this point in the history

  7. Remove RobustScaler (currently does not work with huge datasets)

    louisPoulain committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    5642c0c View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. Fix typo

    @louisPoulain
    louisPoulain authored Jun 19, 2024
    Configuration menu
    Copy the full SHA
    fec0033 View commit details
    Browse the repository at this point in the history

  2. Update default fillvalues and multi.from_dict 

    Remove BoxCox as we have already YeoJohnson (an extension)
    Louis Poulain--Auzéau committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    4cd4a8e View commit details
    Browse the repository at this point in the history

  3. Modify fillvalue to -10. 

    In some cases Yeojohnson transform if less than -5 causing the inverse transform to not be equal
    Louis Poulain--Auzéau committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    2d4111d View commit details
    Browse the repository at this point in the history

  4. Minor modif

    Louis Poulain--Auzéau committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    3a045cf View commit details
    Browse the repository at this point in the history

  5. Modify tests. Tolerance is set to 1e-6 for the inverse transform test

    Louis Poulain--Auzéau committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    34cb53e View commit details
    Browse the repository at this point in the history

  6. Remove debugging code

    Louis Poulain--Auzéau committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    9464260 View commit details
    Browse the repository at this point in the history

  7. Modify assert with correct fillvalue

    Louis Poulain--Auzéau committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    fc5d5c7 View commit details
    Browse the repository at this point in the history

  8. Bug fix: in some cases, transform seemed to modify in-place the input… 

    … dataset
    Louis Poulain--Auzéau committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    bb0a47a View commit details
    Browse the repository at this point in the history

  9. Ensure fillvalue is -5 in this test

    Louis Poulain--Auzéau committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    79d990b View commit details
    Browse the repository at this point in the history

  10. Modify floating point precision

    Louis Poulain--Auzéau committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    6ad1c47 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. Remove RobustScaler and YeoJohnson, refactor the classes into a norma… 

    …lization class handling all techniques to normalize and a normalizer (multinormalizer)
    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    804bc0b View commit details
    Browse the repository at this point in the history

  2. Change gitignore

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    857dcd1 View commit details
    Browse the repository at this point in the history

  3. Change code according to the new way normalizer are being defined

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    edad113 View commit details
    Browse the repository at this point in the history

  4. Remove unused function

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    981e899 View commit details
    Browse the repository at this point in the history

  5. Change code according to the new normalizer definition

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    a825330 View commit details
    Browse the repository at this point in the history

  6. Changes according to standardizers.py

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    88b7bf5 View commit details
    Browse the repository at this point in the history

  7. Changes according to standardizers.py

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    4f15068 View commit details
    Browse the repository at this point in the history

  8. Rename the test to rpevent it from being launched by pytest (test to … 

    …be removed)
    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    8750db9 View commit details
    Browse the repository at this point in the history

  9. Modify code to work with the new normalizers

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    2ba7abc View commit details
    Browse the repository at this point in the history

  10. Clean a bit the code

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    eb3e8be View commit details
    Browse the repository at this point in the history

  11. Update doc, remove print messages

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    d5629b0 View commit details
    Browse the repository at this point in the history

  12. Update test

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    7866d46 View commit details
    Browse the repository at this point in the history

  13. Remove import

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    67ddcc6 View commit details
    Browse the repository at this point in the history

  14. Modify the default normalization handling

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    745f843 View commit details
    Browse the repository at this point in the history

  15. Add fitting of data

    louisPoulain committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    b3b2e37 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. Minor simplification

    louisPoulain committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    a3ae2fb View commit details
    Browse the repository at this point in the history

  2. Change names to reflect more accurately the roles of the classes

    louisPoulain committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    1cfd1b7 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Revert to use "normalizer" as input name 

    See #43 (comment)
    louisPoulain committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    5f76a3e View commit details
    Browse the repository at this point in the history

  2. Use again as pytesting. Removed the serialization test as serializati… 

    …on was moved to the data_transformer class
    louisPoulain committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    0edf87d View commit details
    Browse the repository at this point in the history

  3. Rename the method according to #43 (comment)

    louisPoulain committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    1a7129a View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Add class to handle value filling

    louisPoulain committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    a8ce0db View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. Merge branch 'main' into feature/ValueFiller

    Louis Poulain--Auzéau committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    4774838 View commit details
    Browse the repository at this point in the history

  2. Add some initial thoughts and rename value_filler.py to imputer.py

    @louisPoulain
    louisPoulain authored Oct 18, 2024
    Configuration menu
    Copy the full SHA
    5d53eb9 View commit details
    Browse the repository at this point in the history

  3. Prepare a template for imputation methods

    @louisPoulain
    louisPoulain authored Oct 18, 2024
    Configuration menu
    Copy the full SHA
    00c203d View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. (WIP) Prepare PersistentImputation

    @louisPoulain
    louisPoulain authored Oct 22, 2024
    Configuration menu
    Copy the full SHA
    4fff489 View commit details
    Browse the repository at this point in the history

  2. (WIP) Finish PersistentImputation

    @louisPoulain
    louisPoulain authored Oct 22, 2024
    Configuration menu
    Copy the full SHA
    da26df6 View commit details
    Browse the repository at this point in the history

  3. Improve classes, finish dataImputation super-class

    @louisPoulain
    louisPoulain authored Oct 22, 2024
    Configuration menu
    Copy the full SHA
    5d2f1fa View commit details
    Browse the repository at this point in the history

  4. Create DataImputer on the model of DataTransformer, left important TO… 

    …DOs to address
    @louisPoulain
    louisPoulain authored Oct 22, 2024
    Configuration menu
    Copy the full SHA
    7a4808a View commit details
    Browse the repository at this point in the history

  5. Small text fixes

    @louisPoulain
    louisPoulain authored Oct 22, 2024
    Configuration menu
    Copy the full SHA
    53bedd9 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. Update ConstantImputer

    @louisPoulain
    louisPoulain authored Oct 23, 2024
    Configuration menu
    Copy the full SHA
    0a17fa5 View commit details
    Browse the repository at this point in the history

  2. Delete tests/test_data_transformer.py

    @louisPoulain
    louisPoulain authored Oct 23, 2024
    Configuration menu
    Copy the full SHA
    855d6ca View commit details
    Browse the repository at this point in the history