Skip to content

MediaPipe v0.10.10
Compare
Choose a tag to compare
@ayushgdev ayushgdev released this 23 Feb 06:25
· 985 commits to release since this release
v0.10.10
d2bc9e5

Build changes

  • Fix TensorsToSegmentationCalculator gpu dependencies.
  • Open Source build rules for quantization_util
  • Added the binary the converter factory to run the model weight conversion.
  • Integrates the kMemoryManagerService into ImageToTensorCalculator and InferenceCalculatorDarwinn.
  • Updated iOS OpenCV source build to exclude highgui and videoio
  • Open source some BUILD rules for Converter package

Framework and core calculator improvements

  • Added Face Landmarker C Tasks API and tests
  • Added Pose Landmarker C Tasks API
  • Use memcpy now for copying data and indicate how the data is stored
  • Remove superfluous glFlush().
  • Added Face Detector C Tasks API
  • Add mediapipe::file::IsDirectory helper
  • Deprecate ImageFrame::ByteDepth
  • Added files for the Image Segmenter C Tasks API
  • Add general support for PathToResourceAsFile to TfLiteModelLoader
  • Add CalculatorGraph::SetErrorCallback to receive errors in case of async graph use cases.
  • Add JAX as requirements for MediaPipe python package
  • Introduces HardwareBufferPool based on the ReusablePool and MultiPool
  • Added the base classes for the LLM weight converter.
  • Add stdbool import to C API
  • Introduces MemoryManagerService with HardwareBufferPool and integrates it into the Tensor class.
  • Added the model writer that writes to the weight binary files.
  • Fix GlContext (attachments) cleanup in case of a failing GlContext initialization.
  • Add option for using variable XNNPACK operators to MediaPipe XNNPACK flags
  • Make InferenceCalculatorDarwinn support float and int32 as input data type.
  • Adds VectorToTensorCalculator
  • Enable HardwareBufferPool only if MEDIAPIPE_TENSOR_USE_AHWB is enabled
  • Enables MultiPool and ReusablePool to pass on absl::Status returns originating from object factory methods.
  • Adding TENSOR to InferenceCalculatorCpu to remove vector encumbrances
  • Update Clang to version 16
  • Add ability to preserve output format to GlScalerCalculator
  • Add explicit depedency on XNNPACK & cpuinfo
  • Update TensorFlow and Android NDK dependency
  • Adds MEDIAPIPE_ANDROID_LINK_NATIVE_WINDOW condition to hardware_buffer_android
  • Support interpolate flags in image_to_tensor_converter_opencv.
  • Add MODEL_VIEW side input to tflite_model_calculator

MediaPipe Tasks update

This section should highlight the changes that are done specifically for any platform and don't propagate to other platforms.

Android

  • Added HolisticLandmarker
  • Migrate TextGenerator Java API to C Wrapper
  • Add LlmTaskRunner to TextGenerator sources
  • Don't cache the JNI environment for async calls
  • Simplified api interface
  • Updates to LLM JNI Layer
  • Handle model loading on Android
  • Support custom cache dir.
  • Pass cacheDir to LLM engine from Java API
  • Add "done" field to the Java LLM API
  • Removed backend option from JNI layer

iOS

  • Updated supported pixel formats in iOS image classifier Documentation
  • Removed support for CVPixelBuffer of type 32RGBA
  • Added support for creating CVPixelBuffer from C++ Images to iOS MPPImage Utils
  • Updated implementation of MPPImage Utils to reduce lines of code
  • Added iOS interactive segmenter options, helpers, implmentation and basic tests
  • Added iOS Image Embedder API
  • Enabled stream mode on iOS pose landmarker
  • Fixed issue with iOS Language Detector Prediction Count
  • Added iOS language detector to Cocoapods build
  • Added cosine similarity method to iOS MPPImagEmbedder
  • Updated method signature of MPPImageEmbedderResult initializer
  • Added packet validation in MPPImageEmbedderResult+Helpers
  • Added tests for creating MPPImage with source type UIImage from C++ Image
  • Renamed methods in MPPImageUtilsTests
  • Added a new class for iOS Interctive Segmenter Results
  • Added iOS interactive segmenter to cocoapods build
  • Added method for initializing MPPImages of all source types from MPPImage+TestUtils
  • Added support for creating MPImages of sample buffer source type from C++ Images in MPPImage+Utils
  • Added provision to initialize MPImages of source type sample buffer to MPImage+TestUtils
  • Added tests for initialization of MPImages with source type sample buffer from MPImage+Utils
  • Updated documentation of iOS hand landmarker, image embedder, image segmenter, interactive segmenter, object detector
  • Added provision to create graph config from task options that use any proto for iOS tasks
  • Added new methods to MPPTaskOptionsProtocol.h
  • Updated iOS task runner to initialize tasks using MPPTaskInfo
  • Updated iOS text task runner to initialize tasks from task Info.
  • Updated iOS vision task runner to use new methods from MPPTaskRunner
  • Updated MPPTaskOptionsProtocol
  • Updated MPPTaskRunner initializer
  • Fixed iOS framework conflicts with TensorFlowLiteC and OpenCV CocoaPods
  • Fixed issue in installing iOS tasks text and vision libraries in a single project

Javascript

  • Extend verifyGraph to be compatible with proto3.
  • Add Holistic Landmarker Web API
  • Add export declarations to PoseLandmakerResult
  • TypeScript: adding VideoFrame typings support to video input
  • Guard WaitOnGpu with extra OpenGL checks.
  • Explicitly cast at callsite of WebGL context creation to avoid compilation errors with newer Emscripten versions.

Python

  • Added Holistic Landmarker Python API
  • Support both proto2 and proto3 in task subgraph options configuration, and revised the Holistic Landmarker API's implementation
  • Update holistic_landmarker.py
  • Documented HolisticLandmarker
  • Fixing delegate passing argument in BaseOptions
  • Add model_ckpt_util to Python Build script
  • Use pybind_library for GenAI Converter build

MediaPipe Dependencies

  • Expose MediaPipe's ABSL and Sentencepiece as shared dependencies
  • Remove Sentencepiece's LOG function
  • Removed unwanted headers from opencv_ios_xcfraemworl_files.bzl
  • Update WASM files for 0.10.10 release
  • Upgrade TypeScript to 5.3.3
Assets 2
Loading