I had a similar situation when I was migrating data off an old FTP for a client. Downloading wasn’t realistic with the size of the files, so I looked into ways to connect
google drive to ftp directly. What worked for me was setting them up side by side in Finder so I could move folders between them without storing anything locally. It handled big transfers smoothly, and I didn’t have to babysit the process like I usually do with manual uploads.