thoughts

• A reduction approach to mechanistic interpretability (MI)
  • Borrow the idea from John Langford’s reductions webpage. Maybe, I should read the several papers that use the concept of reduction to solve complex learning problems, such as reductions for fair classification.
  • Watch the video by John et al and the tutorial slides.

paper-buf

Top-10 paper of the day to read

• LLM safety related
  • Open problems in machine unlearning for ai safety
  • Safetywashing: Do AI safety benchmarks actually measure safety progress?
• Reasoning
  • Search-o1: Agentic search-enhanced large reasoning models
  • rStar-Math: Small llms can master math reasoning with self-evolved deep thinking
    • Benchmarks:
      • MATH
      • AIME 2024
      • Olympiad Bench
      • College Math
      • Omni-Math
  • Towards system 2 reasoning in llms: Learning how to think with meta chain-of-thought
• Others LLM related
  • Demystifying domain-adaptive post-training for financial llms, domain-specific llms
  • Long context vs. rag for llms: An evaluation and revisits, knowledge update (rag or long prompt?)
  • Analyzing memorization in large language models through the lens of model attribution
  • Decoding knowledge in large language models: A framework for categorization and comprehension
• Machine learning
  • An analytic theory of creativity in convolutional diffusion models

algorithm-practice-in-industry

• https://github.com/Doragd/Algorithm-Practice-in-Industry: A github repo that holds well-recognized resources about Industry algorithm practices.

learn a little more about spectral clustering

• The wikipedia page

“In multivariate statistics, spectral clustering techniques make use of the spectrum (eigenvalue) of the similarity matrix”

The algorithm can be described as:

1. Calculate the Laplacian L (or the normalized Laplacian)
2. Calculate the first k eigenvectors
3. Consider the matrix formed by the first k eigenvectors, the l-th row defines the features of the graph node l
4. Cluster the graph nodes based on these features (e.g. using k-means clustering)

combinatorics

$$ C_N^2 = \sum_i C_{K_i}^2, \text{ where } \sum_i K_i = N $$

The above equation is NOT hold! How to enhance my knowledge about combinatorics? Refer to the notes on Combinatorics Page.