Step 4 - Convert the json files to Vectors Use the file preprocess_json to convert the json files to a dataframe with Embeddings and save it as a joblib pickle ...

The BigQuery table is too large to fit into memory, so you need to write a generator called query_bigquery_chunks to yield chunks of the dataframe for processing. Additionally, an extra column ...