Notable technical projects

Highlight detection in soccer videos

GLIACloud, 2018

• Built a simple CNN + RNN model which was trained on labeled soccer videos from Chinese Super League.

Distillation of CLIP model

PicCollage, 2021

• Distilled the ViT component of OpenAI CLIP model (~ 24MB with FP16 precision), which was then deployed on edge and is currently in use by multiple apps.
  • Some details are available here.
  • To the best of my knowledge, this was the first distilled CLIP model. Others followed later, most notably MobileCLIP by Apple.

Transition detection in videos

PicCollage, 2021

• This was done by looking at the spectral properties of the cosine similarity matrix of the video frames (Spectral Graph Theory to the rescue!).
  • Led to the first patent applied for by PicCollage.

Training a GNN for photo to sticker recommendation

PicCollage, 2022

• Implemented a variant of CompGCN.
• The most important change was that our model was inductive while the original implementation was transductive. This was done by:
  1. fixing the initial node embedding to be a CLIP embedding, and
  2. learning a transformation to map the initial node embedding instead of changing the node values directly.

Bringing Generative AI related technologies to build products/features on

PicCollage, 2022

• Set up training pipelines for DreamBooth, LoRA and ControlNet.