Skip to content

stepi vs lossi plot bug fixed in makemore2 mlp notebook#10

Open
ibadrather wants to merge 23 commits intokarpathy:masterfrom
ibadrather:master
Open

stepi vs lossi plot bug fixed in makemore2 mlp notebook#10
ibadrather wants to merge 23 commits intokarpathy:masterfrom
ibadrather:master

Conversation

@ibadrather
Copy link
Copy Markdown

To keep track of training steps and loss you initialised lossi and stepi lists. Then while running the training loop multiple times loss was appended to lossi correctly, but you were appending i to stepi and that restarting from 0 again. Hence when you ran the training loop multiple times in the video, the loss was appended but with duplicate steps.

####################################
To keep track of actual steps I did:
step = 0 # initialise

for i in range(50000):
step += 1
stepi.append(step)

instead of:
for i in range(50000):
stepi.append(i)

#####################################

I have uploaded the plots with incorrect and correct step tracking. I ran the training loop 2 times in both cases with 50000 training steps each time. But one plot is only showing 50000 steps instead of 100000 steps.

@ibadrather
Copy link
Copy Markdown
Author

These are the plots I am talking about. Both have been run for 100000 epochs

loss_with_correct_step
loss_with_wrong_step

Sorry for a messy pull request I am still learning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants