Indeed, the problem was with nginx configuration, not with React. I was missing a directive to tell nginx to ignore the 'app' part of the URL, and to load files from a different directory. Doing the following solved the problem:
location /app { alias /usr/share/nginx/html; try_files $uri $uri/ /app/index.html =404; }