Fix broken base_config for RL training.

[niting]

Broken by #1375 which now requires a dataset to be specified.

Resolves #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

Reference

Colab Notebook

Checklist

• I have added all the necessary unit tests for my change.
• I have verified that my change does not break existing code and all unit tests pass.
• I have added all appropriate doc-strings/documentation.
• My PR is based on the latest changes of the main branch (if unsure, rebase the code).
• I have signed the Contributor License Agreement.
• I have followed Contribution Guidelines.