Week 4 2019 Spring

Homework:

• Reading:
  • tfjs: Handwritten digit recognition with CNNs
• Coding:
  • Continue working on using Teachable Machine Prototype to build your own image/speech classifier, update your link on week 3 homework page.
  • Read the code of 3 doodle classifier examples, play with the demo, think about the difference between the 3 examples.
• Writing:
  • Write a final assignment proposal on your blog. Read more about the final assignment proposal below. Add your blog link below.

Doodle Classifier Examples

• (1) Doodle Classifier on 100 classes, code: pre-trained model
• (2) Doodle Classifier with KNN Classifier, code: transfer learning
• (3) Train your own Doodle Classifier, code: create our own model

Resource:

• Coding train: Doodle Classifier - Intelligence and Learning
• Convolutional Neural Networks - The Math of Intelligence

Final Assignment Proposal

The final assignment is a 3-week assignment that includes project proposal and development. The final assignment presentations will be on week 7(May. 8).

The final assignment could be any creative project that builds off of or is inspired by the concepts we've covered in this class. Feel free to think non-traditionally. It doesn't need to be a web app, you can also add physical computing elements to it. You can collaborate with anyone in any class, or make it as a part of a larger project integrated with a different class.

The main idea is to build a simple but working application that demonstrates your idea and expresses your creativity.

Some suggestions:

• Start with a simple idea, do some experiments with it, and then add more features to it. Keeping things simple and small in scope is a plus. If your project idea is a big one, consider documenting the larger idea but implementing just a small piece of it.
• If you get stuck on the code or other technical issues, ask for help in advance from residents and teachers.
• If your project involves training a model, leave some extra time for collecting the data and training.
• Leave some time for documentation. The live demo doesn't always work, you could record a video and make some screenshots in case the demo breaks at the last minute.

Get started:

You can start your project proposal by thinking about -

• What model do you like the best? any model from ml5js and tfjs.
• What are your inspirations: How did you become interested in this idea? quotes, photographs, products, projects, people, music, political events, social ills.
• Are there some existing, related or similar project: drawings, images, videos, sounds, text
• Are there some code examples that might be helpful: your own sketches and/or other people's sketches.
• What is your project title?
• How do you describe your project in one sentence?
• The context of your project? Who's it for? How will people experience it? Is it interactive? Is it practical? Is it for fun? Is it emotional? Is it to provoke something?
• Do you have any questions, or need any help? Conceptually and technically.

Some sample ideas to get you started:

If you have trouble coming up with ideas, here are some sample ideas to get you started.

• Image classifier with speech output using MobileNet and p5.speech. See code and demo in week 2.
• Image classifier with translated results and speech output using MobileNet, Google Translate API and p5.speech. See code and demo in week 2.
• Webcam Scavenger Hunt Game using MobileNet, p5.js, p5.speech. See code and demo in week 2.
• Two-player scavenger hunt(combine with socket.io)
• Using PoseNet to play/pause music. See code and demo in week 2.
• PoseNet dancing music generator
• Using KNN Image Classifier to move a square up/down, left/right on the canvas. See code and demo in week 2.
• Use KNN Image classifier to play Pong game, snake game with a webcam
• Speech Assistant, KNN Image classifier + p5.speech.js
• KNN Classification on Webcam Images with Speech Output with p5.speech.js. See code and demo in week 2.
• Training a KNN Classifier to turn on/off a Philips hue lamp.
• tf.js Speech commands, use speech command to control a game
• Speech command app using p5 speech. See code and demo in week 6.
• Style Transfer mirror installation
• Pix2pix drawing app
• Doodle Classifier. See code and demo in week 3.
• SketchRNN drawing app. See code and demo in week 6.
• Body Pix

Inspirations:

• Google AI experiments
• tf.js Gallery
• ml5.js experience
• Open AI Scholars Class of ’18: Final Projects
• Distill

Resources:

• ml5.js examples
• tf.js examples
• tf.js models

Questions?

• add your question here

Add your name/link to your final proposal below

• Your name, [Project name](blog Link), Project Description