Capistrano deployment - Sphinx index step very SLOW after updating listing_index.rb

Recently we modified the Sharetribe code to allow for user custom fields in addition to listing custom fields. We also display user fields in listing pages, so that the viewer knows more about the user, like this listing:
https://www.freelensia.com/listings/422145

As part of our PR, we update the listing_index.rb like this:

After this PR is merged, our Capistrano code becomes very SLOW at this step:
bundle exec rake ts:index

We tried 3-4 times and all of the time the process runs over night with completing.

Our team engineer suggests that we must add a ts:rebuild step before the ts:index step. We tried this and the process is reduced to about 45 minutes but still not acceptable for the small size of our project.

Anybody with ideas?