About the correspondence between the phases proposed in the paper and the phases in the code

[luhexin]

I appreciate your contribution, and I've been studying this paper recently, but have a few questions

Question 1：
In the paper，pipeline of proposed few-shot learning method, including three phases:
(a) DNN training on large-scale data
(b) Meta-transfer learning
(c) meta-test

In README.md you mention the pre-train phase
meta-train phase and meta-test phase

Is the pre-training phase equivalent to DNN training on large-scale data?
meta-train phase = Meta-transfer learning?
meta-test phase = meta-test?

Question 2：

meta-transfer-learning/pytorch/trainer/meta.py

Lines 239 to 294 in 835b6bb

	def eval(self):
	"""The function for the meta-eval phase."""
	# Load the logs
	trlog = torch.load(osp.join(self.args.save_path, 'trlog'))
	# Load meta-test set
	test_set = Dataset('test', self.args)
	sampler = CategoriesSampler(test_set.label, 600, self.args.way, self.args.shot + self.args.val_query)
	loader = DataLoader(test_set, batch_sampler=sampler, num_workers=8, pin_memory=True)
	# Set test accuracy recorder
	test_acc_record = np.zeros((600,))
	# Load model for meta-test phase
	if self.args.eval_weights is not None:
	self.model.load_state_dict(torch.load(self.args.eval_weights)['params'])
	else:
	self.model.load_state_dict(torch.load(osp.join(self.args.save_path, 'max_acc' + '.pth'))['params'])
	# Set model to eval mode
	self.model.eval()
	# Set accuracy averager
	ave_acc = Averager()
	# Generate labels
	label = torch.arange(self.args.way).repeat(self.args.val_query)
	if torch.cuda.is_available():
	label = label.type(torch.cuda.LongTensor)
	else:
	label = label.type(torch.LongTensor)
	label_shot = torch.arange(self.args.way).repeat(self.args.shot)
	if torch.cuda.is_available():
	label_shot = label_shot.type(torch.cuda.LongTensor)
	else:
	label_shot = label_shot.type(torch.LongTensor)
	# Start meta-test
	for i, batch in enumerate(loader, 1):
	if torch.cuda.is_available():
	data, _ = [_.cuda() for _ in batch]
	else:
	data = batch[0]
	k = self.args.way * self.args.shot
	data_shot, data_query = data[:k], data[k:]
	logits = self.model((data_shot, label_shot, data_query))
	acc = count_acc(logits, label)
	ave_acc.add(acc)
	test_acc_record[i-1] = acc
	if i % 100 == 0:
	print('batch {}: {:.2f}({:.2f})'.format(i, ave_acc.item() * 100, acc * 100))
	# Calculate the confidence interval, update the logs
	m, pm = compute_confidence_interval(test_acc_record)
	print('Val Best Epoch {}, Acc {:.4f}, Test Acc {:.4f}'.format(trlog['max_acc_epoch'], trlog['max_acc'], ave_acc.item()))
	print('Test Acc {:.4f} + {:.4f}'.format(m, pm))

Does the above code correspond to “classifier fine-tuning” in the (c)meta-test phase?
Line 258 in this code sets the model to eval mode and does not fine-tune the base-learner

Question 3：
Data set in the code is divided into three parts: training set, validation set and test set.
Are the train samples in (a), the meta-batches in (b) and the train samples in (c) sampled from the training set?
Are the test samples in (c) sampled from the test set?

[yaoyao-liu]

Hello @luhexin

Thanks for your interest in our work. The answers to your questions are as follows:

• Answer to Q1: Yes.
• Answer to Q2: The base learner is fine-tuned by the following line:

  meta-transfer-learning/pytorch/trainer/meta.py

  Line 283 in 835b6bb

  logits = self.model((data_shot, label_shot, data_query))

• Answer to Q3: For example, on miniImageNet, the model is trained on the meta-train set, including pre-training and meta-transfer learning. Then, the model is evaluated on the meta-test set. The validation set is never used. This is to align with other related work, e.g., MAML and ProtoNets.

If you have any further questions, please feel free to contact me.

Best,

Yaoyao

[luhexin]

@yaoyao-liu Thanks for your reply. I'm still confused about some things.

Q1.In the phase of classifier fine-tuning, is base learner fine-tuned with test set?

Q2.In the phase of classifier fine-tuning, I locate this function, but I still don't know how it update the parameters of the base learner.
Are the parameters optimized according to loss in this code?

meta-transfer-learning/pytorch/models/mtl.py

Line 84 in 835b6bb

def meta_forward(self, data_shot, label_shot, data_query):

Q3.The final evaluation phase does not update the parameters of the meta-learner and the base learner. Where does this phase correspond to the code?