architecting systems that think - responsibly โข senior ml engineer โข bit of stock market too.
Joined May 2020
- Tweets 954
- Following 100
- Followers 146
- Likes 12,807
284 Photos and videos
Pinned Tweet
27 Apr 2025
๐งต How to Become a #MachineLearning and #AI Expert in 6 Months (Detailed, Free Resources)
Ready to dive deep into ML? From my 5 years of experience in ML and AI, here's a month-by-month roadmap using comprehensive, high-quality, and free resources. Let's get started! ๐
1
1
7
2,160
The Information Bottleneck (IB) principle offers a profound lens for understanding why deep networks generalize despite overparameterization. Introduced by Tishby et al. (2015), IB frames learning as an optimal compression problem: a network layer should retain only information in input X that is predictive of label Y.
Formally, IB minimizes:
L = I(X; T) - ฮฒยทI(T; Y)
where I(ยท;ยท) denotes mutual information, T is the learned representation, and ฮฒ controls the compression-prediction tradeoff. The optimal solution satisfies:
p(t|x) โ p(t)ยทexp(-ฮฒยทD_KL[p(y|x) || p(y|t)])
During SGD training, neural networks exhibit two distinct phases:
1. ๐
๐ข๐ญ๐ญ๐ข๐ง๐ ๐ฉ๐ก๐๐ฌ๐: I(X;T) and I(T;Y) both increase rapidly as the network memorizes patterns
2. ๐๐จ๐ฆ๐ฉ๐ซ๐๐ฌ๐ฌ๐ข๐จ๐ง ๐ฉ๐ก๐๐ฌ๐: I(X;T) decreases while I(T;Y) remains stable-the network forgets irrelevant input details while preserving task-relevant signals
The critical insight: ๐ ๐จ๐จ๐ ๐ ๐๐ง๐๐ซ๐๐ฅ๐ข๐ณ๐๐ญ๐ข๐จ๐ง ๐๐จ๐ซ๐ซ๐๐ฅ๐๐ญ๐๐ฌ ๐ฐ๐ข๐ญ๐ก ๐๐จ๐ฆ๐ฉ๐ซ๐๐ฌ๐ฌ๐ข๐จ๐ง. Layers progressively discard nuisance information (noise, spurious correlations) irrelevant to Y. Empirically, the penultimate layer's I(X;T) correlates strongly with test accuracy.
For a k-class problem, the representation T at the final hidden layer satisfies I(T;Y) โค H(Y) โค log k bits, bounding retained label information. Stochastic gradient noise implicitly implements a form of variational optimization over this constrained objective.
Recent work (Saxe et al., 2018) nuances this: without nonlinearities or specific activation symmetries, networks don't always compress. However, with ReLU-like activations and finite precision, compression emerges naturally as an implicit regularizer.
This explains a paradox: massive networks (millions of parameters) generalize well because they learn ๐ฆ๐ข๐ง๐ข๐ฆ๐๐ฅ ๐ฌ๐ฎ๐๐๐ข๐๐ข๐๐ง๐ญ ๐ซ๐๐ฉ๐ซ๐๐ฌ๐๐ง๐ญ๐๐ญ๐ข๐จ๐ง๐ฌ-encoding exactly what Y requires and nothing more, effectively solving an information-theoretic optimization that classical statistical learning theory struggles to characterize.
#InformationBottleneck #DeepLearningTheory #Generalization
18
Mohit ๐ผ retweeted
1. Never tell anyone how much you make
2. Get proper sleep (or at least try to do so)
3. Live in a way no one else does (do the opposite)
4. Read every day
5. Learn to communicate
6. Go off the radar for 48 hours
7. Use AI the right way
8. You are one DM away from changing your life
9. Find a stress protocol that works for you
10. Learn how to communicate your emotions (#1 way not to get depressed)
11. Always minimize (less is more)
12. Phone calls > podcasts, audiobooks, and music
13. Change your workspace often (Creativity hack)
14. Spontaneity
12
60
656
28,728
May 28
uh here we go again.
Introducing Claude Opus 4.8: it builds on Opus 4.7 with sharper judgment, more honesty about its own progress, and the ability to work independently for longer than its predecessors.
Available today at the same price.
ALT Benchmark table showing how Claude Opus 4.8 compares to its predecessor and to other models on tests of coding, agentic skills, reasoning, and practical knowledge work tasks.
36
Mohit ๐ผ retweeted
May 27
> lifted 30 crore indians out of poverty
> free ration to 80cr poor people
how both things work simultaneously?
May 27
What's the biggest scam in history?
69
381
2,929
54,755
May 26
Is a 60-day notice period really that big of a deal for recruiters now?
Hadd ho gayi - the moment they hear โ60 daysโ they donโt even listen to the next word.
Pareshaan ho gaya bhai.
25
May 24
I never really understood why people call their newly bought gadgets or vehicles a โnew toy.โ
For someone coming from a lower middle-class background, itโs never just a toy. It carries years of hard work, sacrifices, patience, and silent prayers behind it.
Every small achievement feels deeply personal because you know what it took to reach there.
Grateful. Blessed. Proud. โค๏ธ
Definitely not a toy. ๐งฟ
6
18
2,328
May 24
Reading the comments made me realize how many people started watching cricket after COVID and still donโt understand the game.
Anyway - Arjun bowled really well. ๐ฅ
May 23
Well done, Arjun. โค๏ธ
Proud of the way youโve carried yourself through this season, always believing in your ability, staying patient, working hard quietly, and remaining positive despite having to wait for your opportunity till the very last match.
Cricket tests patience as much as skill, and you handled both beautifully today.
Keep your feet on the ground, and continue being in love with the game like you always have.
Love you always.๐
109
May 21
Didnโt know Google pays this much less for an L4 role in India.
Saved myself the effort of preparing and going through another switch cycle. ๐ฅฒ
2
5
4,359
May 10
เคเค เคฎเคพเค เคธเฅ เคฌเคกเคผเคพ เคฏเฅเคฆเฅเคงเคพ เคเฅเค เคจเคนเฅเค เคนเฅเคคเคพ โฅ๏ธ
May 10
She lost her husband in an accident.
Now sheโs working as a Zomato delivery worker to feed her family.
She even takes her two kids with her during deliveries.
Respect to such brave women.
She is a million times better than feminist girls like Rebel Kid Apoorva.
102
May 2
Lagrangian formulations for implicit neural representations and solver theory-
The Lagrangian formulation of implicit neural representations (INRs) reveals a deep connection between neural networks and classical mechanics-unifying network architecture with variational principles.
An INR parameterized by weights ฮธ represents a continuous signal as f(x; ฮธ), where x โ โแต. The Lagrangian approach introduces a functional:
โ[ฮธ] = โซ_ฮฉ L(x, ฮธ, โฮธ, โยฒฮธ...) dx
Minimizing this action S = โซ โ dt yields Euler-Lagrange equations: โโ/โฮธ - โ ยท (โโ/โโฮธ) = 0
This connects to solver theory: training INRs is equivalent to solving a boundary value problem. The network dynamics follow:
M(ฮธ)ฮธฬ D(ฮธ)ฮธฬ = -โV(ฮธ)
where M is the mass matrix (parameter metric) and V captures reconstruction loss.
Key insight: symplectic integrators preserve the symplectic structure of (ฮธ, โโ/โฮธ), ensuring long-term stability-critical for physics-informed INRs.
Alternatively, treating ฮธ as generalized coordinates gives the Hamiltonian: แน = -โV(ฮธ), ฮธฬ = Mโปยนp, enabling geometrically stable optimization via reversible dynamics solvers like Ritter et al.'s port-Hamiltonian approach.
This framework connects directly to adaptive solvers, residual networks as discrete symplectic systems, and the geometry of loss landscapes. The action principle provides principled regularization beyond simple Lยฒ reconstruction, with natural energy conservation properties.
#LagrangianMechanics #ImplicitNeural #SolverTheory #MachineLearning
1
3
133
Apr 29
If he plays all 20 overs - almost every bowler will get hit for four sixes in their spell by him.
Crazy kid ๐คฏ
Apr 29
Vaibhav Suryavanshi has smashed 61 sixes from his 290 balls career in the IPL so far. ๐ฅถ
- A six every 4.75 ball. ๐คฏ
108