I HAVE TRAPPED YOU HERE FOR 0 SECONDS
Arkansas State University
Degree comp sci
You can click on this site trust me
LANGUAGES: JavaScript, C++, Rust, Java, Python
TOOLS: Git, Node/Bun, Firebase, MySQL, Pytorch/ML, Cloud, React/Svelte
I wrote AI from scratch
EMAIL: trevorblythe82@gmail.com
PHONE: ask me
LOCATION: not here
github.com/TrevorBlythe
linkedin.com/in/trevor-blythe-31725b225/
The term "imaginary" means nothing to what it actually is.
A complex number is a 2D vector (an arrow) with a different definition of multiplication.
"i" is just complex number pointing up. (x:0, y:1)
Multiplication for complex numbers is defined as:
(a,b) * (c,d) = (ac−bd, ad+bc)
This process:
1. Scales the vector’s length, and
2. Rotates it by vector's angle.
This process is completely arbitrary, mathematicians only did this so they could say i * i = -1, for clout. And that angers me.
people should be cohesive enough to break away from this technical debt.
AI's only understand numbers, so every word has a hard-coded array that represents its meaning. That array is called an "embedding".
Each number in the array, represents a thing about the word. ex: is the word a fruit, is the word red? (this array is really long in reality).
But a words meaning, depends on context. EX: (money bank, river bank).
"self attention" adds relevant context from previous words, to create a contextual embedding that contains the full meaning of the current word.
If we had an embedding, with ONLY the number that represent being an adjective, you could dot product this adjective detector embedding, with any word, to test if its an adjective.
If an AI is given the text "river bank", we need to add the meaning of "is terrain", into "bank"s embedding. We would need to run a "terrain detector" on all previous words. So the AI needs a way of knowing what detectors we need to run depending on the word.
Each AI has three hardcoded 2D arrays called the "query matrix" (Q), "key matrix" (K), and "value matrix" (V). These three matrices, can be used to compute a distinct way context changes meaning.
For each input word, the query and key and value matrix is multiplied by the embedding, producing Q and K and V vectors (Qv, Kv, and Vv).
A words Qv, represents what context it needs. (like is terrain or money related).
A words Kv, represents what context it can give to other words. (like is terrain or money related).
A words Vv represents what to add to other words embedding (like being terrain or a building).
"river bank", "river"s Kv matches with "bank"s Qv, "rivers" Vv added to "bank"
"money bank", "money" Kv matches with "bank"s Qv, "moneys" Vv added to "bank"