Zhening Li

Focusing on deterministic sparse-reward environments, we show theoretically and empirically that RL performance gain from skills (temporal abstractions) is worse in environments where solutions to states are less compressible (in the information-theoretic sense). Further theoretical results suggest that skills benefit exploration more than they benefit learning from existing experience, and that using unexpressive skills such as macroactions may worsen RL performance.

TextReact augments predictive chemistry with texts retrieved from the literature. For a given chemistry input, the retrieval model retrieves relevant texts, and the predictor uses both the original input and retrieved texts to output predictions. On reaction condition recommendation and one-step retrosynthesis, TextReact outperforms state-of-the-art deep-learning chemistry models by 58.4% and 13.6–15.7%, respectively.

MolScribe translates molecular diagrams in image format to a structured graph format. It explicitly predicts atoms and bonds along with their positions, and flexibly incorporates symbolic chemistry constraints to recognize chirality and expand abbreviated structures. MolScribe achieves 76–93% accuracy on public benchmarks.

Learning Mathematical Abstractions (LEMMA) learns a hierarchy of abstractions to enhance RL for mathematical reasoning. It augments Expert Iteration with an abstraction step, where solutions found so far are rewritten in terms of new higher-level actions, which then become available to solve new problems. LEMMA increases the performance and generalization of an RL agent on equation solving and fraction simplification.