You might experience this issue if you have assigned one earbud (either L or R) to the Google Assistant function via the Headphones Connect app.
It's important to remember that you can only independently use the earbud that has been assigned to Google Assistant, while the other earbud is tucked away in the charging case.
The other way around does not work because the other (unassigned) earbud cannot be used as a single set. When you place back the Google Assistant earbud in the charging case, it terminates the Bluetooth connection with the playback device, leaving the other earbud unable to establish a connection.
You can choose which headphone unit (L or R) will be set for the Google Assitant function using the Sony | Headphones Connect app.