Imported from https://github.com/google-research/batch_rl