X(Twitter) / Zhihu / Google Scholar / Github / Huggingface / CV

Leap into that sea of light. 跃入那光那海。

Mika’s Blog | Notion

🤔 About Myself

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/7af8f684-8d20-4c20-a9b6-e6c975b8cb4c/f44c448b-c9eb-445b-8a71-5c84d57223ec/WX20240909-1323252x.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/7af8f684-8d20-4c20-a9b6-e6c975b8cb4c/f44c448b-c9eb-445b-8a71-5c84d57223ec/WX20240909-1323252x.png" width="40px" />

Nijika: Qingyu is an undergraduate student from Zhejiang University, class of 2022. He is pursuing a dual degree in English and Computer Science. He is now a research intern @Hedra.

</aside>

❓What Am I Doing Now?

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/7af8f684-8d20-4c20-a9b6-e6c975b8cb4c/f44c448b-c9eb-445b-8a71-5c84d57223ec/WX20240909-1323252x.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/7af8f684-8d20-4c20-a9b6-e6c975b8cb4c/f44c448b-c9eb-445b-8a71-5c84d57223ec/WX20240909-1323252x.png" width="40px" />

Nijika: His research interests and works lie at:

• Efficiency ♿️: Modern LLMs must prioritize hardware efficiency, support parallelism during training, and maintain reasonable computational costs for inference.
• Transparency 🔍: It is essential for humans to understand why LLMs succeed or fail in specific tasks. </aside>

📑  Selected Publications

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/7af8f684-8d20-4c20-a9b6-e6c975b8cb4c/f44c448b-c9eb-445b-8a71-5c84d57223ec/WX20240909-1323252x.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/7af8f684-8d20-4c20-a9b6-e6c975b8cb4c/f44c448b-c9eb-445b-8a71-5c84d57223ec/WX20240909-1323252x.png" width="40px" />

Nijika: Here check the newest ones!→ 📖 Google Scholar (Recommended)

</aside>

StableMask: Refining Causal Masking in Decoder-only Transformer [ICML'24] Qingyu Yin, Xuzheng He, Xiang Zhuang, Yu Zhao, Jianhua Yao, Xiaoyu Shen, Qiang Zhang

****ArXiv / Github / X(Twitter)

Deeper Insights Without Updates: The Power of In-Context Learning Over Fine-Tuning [EMNLP'24 Findings] Qingyu Yin, Xuzheng He, Chak Tou Leong, Fan Wang, Yanzhao Yan, Xiaoyu Shen, Qiang Zhang

****ArXiv / Github / X(Twitter)

Direct Preference Optimization Using Sparse Feature-level Constraints [Preprint] Qingyu Yin*, Chak Tou Leong*, Hongbo Zhang, Minjun Zhu, Hanqi Yan, Qiang Zhang, Yulan He, Wenjie Li, Jun Wang, Yue Zhang, Linyi Yang

****ArXiv / Github / X(Twitter)

🕤 Experience

Intern:

April 1, 2023 → July 1, 2024 Zhejiang University, ZJUNLP [Research Intern]

August 1, 2024 → December 1, 2024 Hedra Inc. [Research Intern]

July 1, 2024 → October 1, 2024 Westlake University, WestlakeNLP [Research Intern]

Reviewer:

October 1, 2024 ICLR’25 [Reviewer]

December 12, 2024 AAAI’25 KnowFM Workshop [Reviewer]

Invited Talk:

June 1, 2024 AI Times for ICML’24 Video

Music:

I’m now the Vocal Guitarist in LilyPuffer, before that, I was the Rhythm Guitarist in The Wall.