Feast v0.8.0

Implemented enhancements:

• Implement JobService API calls & connect it to SDK #1129 (tsotnet)
• Allow user to specify custom secrets to be mounted on Feast Serving and Feast Core pods #1127 (khorshuheng)
• Allow spark expressions in field mapping during Ingestion #1122 (pyalex)
• Update rest endpoints #1121 (terryyylim)
• Add feature table deletion #1114 (terryyylim)
• Add historical retrieval via job service #1107 (oavdeev)
• Implement list job and get job methods for Dataproc launcher #1106 (khorshuheng)
• Allow entities and features to be updated #1105 (terryyylim)
• Add get_by_id and list_jobs interface to the launcher interface and implement it for EMR #1095 (oavdeev)
• Support redis ssl in feast-serving #1092 (oavdeev)
• Add helm charts for feast jobservice #1081 (tsotnet)
• Terraform cleanup: tags, formatting, better defaults #1080 (oavdeev)
• Update docker-compose for Feast 0.8 #1078 (khorshuheng)
• Configure jupyter env on AWS via terraform #1077 (oavdeev)
• Implement EMR job cancelling #1075 (oavdeev)
• Streaming Ingestion Job supports AVRO format as input #1072 (pyalex)
• Accept Pandas dataframe as input for historical feature retrieval #1071 (khorshuheng)
• Add EMR streaming job launcher #1065 (oavdeev)
• EMR launcher #1061 (oavdeev)
• Add AWS managed kafka config to the example terraform #1058 (oavdeev)
• Feast SDK integration for historical feature retrieval using Spark #1054 (khorshuheng)
• Update GetOnlineFeatures method in sdks #1052 (terryyylim)
• "Start Offline-to-online ingestion" method in Python SDK #1051 (pyalex)
• Adding support for custom grpc dial options in Go SDK #1043 (ankurs)
• CLI command to start/stop/list streaming ingestion job on emr #1040 (oavdeev)
• Update serving service to handle new redis encoding #1038 (terryyylim)
• terraform config for aws #1033 (oavdeev)
• Streaming Ingestion Pipeline with Spark #1027 (pyalex)
• Run offline-to-online ingestion job on EMR #1026 (oavdeev)
• Add redis SSL support to the offline-to-online ingestion job #1025 (oavdeev)
• Dataproc and Standalone Cluster Spark Job launcher #1022 (khorshuheng)
• Pyspark job for feature batch retrieval #1021 (khorshuheng)
• Batch Ingestion Job rewritten on Spark #1020 (pyalex)
• Add Feature Tables API to Core & Python SDK #1019 (mrzzy)
• Introduce Entity as higher-level concept #1014 (terryyylim)

Fixed bugs:

• Fix stencil client serialization issue #1147 (pyalex)
• Deadletter path is being incorrectly joined #1144 (pyalex)
• In Historical Retrieval (SDK) use project from client context #1138 (pyalex)
• Pass project from context to get entities #1137 (pyalex)
• JobService is in crashloop after installing helm chart #1135 (pyalex)
• Fix env var names for jupyter terraform config #1085 (oavdeev)
• Fix java class name validation #1084 (oavdeev)
• Multiple tiny AWS related fixes #1083 (oavdeev)

Merged pull requests:

• Make created_timestamp property optional in KafkaSource #1146 (pyalex)
• In Streaming E2E Test filter kafka consumers by group id prefix #1145 (pyalex)
• Limit concurrency on e2e test runs to 1 #1142 (oavdeev)
• Update prow trigger for AWS #1139 (oavdeev)
• e2e test fixes to make them work on AWS #1132 (oavdeev)
• Add feature table name & job id to deadletter destination #1143 (pyalex)
• Drop hardcoded FEAST_CORE_URL env from JobService helm chart #1136 (pyalex)
• Add Prow to AWS codebuild trigger #1133 (oavdeev)
• Optional IngestionJob parameters passed by Spark Launcher #1130 (pyalex)
• Wait for job to be ready before cancelling #1126 (khorshuheng)
• Ability to run e2e tests in non-default project #1125 (pyalex)
• Ensure job is completed when ingesting to BQ #1123 (terryyylim)
• Add end-to-end Prow Job launcher for AWS tests #1118 (woop)
• Add confluent kafka installation to minimal notebook #1116 (woop)
• Scaffolding for integration tests #1113 (khorshuheng)
• Add serving integration test for updated feature type #1112 (terryyylim)
• In Historical Retrieval from BQ join between source & entities is performed inside BQ #1110 (pyalex)
• Make created timestamp column optional #1108 (khorshuheng)
• Add validation when feature type is changed #1102 (terryyylim)
• E2E flow in prow is working #1101 (pyalex)
• Return e2e back to prow runner #1100 (pyalex)
• Add gh workflow for dockerhub #1098 (terryyylim)
• Make demo notebook work on AWS #1097 (oavdeev)
• Update Kubernetes setup for Feast 0.8 #1096 (khorshuheng)
• Refactored end-to-end tests fully orchestrated by pytest #1094 (pyalex)
• Disable statsd by default for spark #1089 (oavdeev-tt)
• Restructure tutorial subfolder #1088 (terryyylim)
• Remove load test from github action #1087 (khorshuheng)
• Add more explanations to the demo notebook, use local file system instead of GCS by default #1086 (khorshuheng)
• Tutorial (Full demo) in Jupyter #1079 (pyalex)
• Add unit test for historical retrieval with panda dataframe #1073 (khorshuheng)
• Remove outdated tutorials #1069 (terryyylim)
• Add method to add feature to Feature table #1068 (terryyylim)
• Historical feature retrieval e2e test #1067 (khorshuheng)
• Use RedisKeyV2 as key serializer and java murmur implementation in Redis Sink #1064 (pyalex)
• Use existing staging client for dataproc staging #1063 (khorshuheng)
• Cleanup CLI and Python dependencies #1062 (terryyylim)
• Refactor Spark Job launcher API #1060 (pyalex)
• Create empty Job Service #1059 (tsotnet)
• Replace Data Source specific format options with DataFormat message #1049 (mrzzy)
• Add created_timestamp_column to DataSource. Rename timestamp_column -> event_timestamp_column #1048 (pyalex)
• Cleanup e2e and docker-compose tests #1035 (terryyylim)
• Add svc account volume mount to prow jobs #1034 (terryyylim)
• Update prow config and makefile #1024 (terryyylim)
• Refactor Python SDK to remove v1 concepts #1023 (terryyylim)