Now, let's write a function to store documents in vector store.
Define a function store_docs that takes url as input, retrieves data from the url website, processes it into document chunks, using the functions we defined earlier and then stores them in a vector store:
def store_docs(url):
text, metadata = get_data_from_website(url)
text = clean_text(text)
docs = text_to_docs(text, metadata)
vector_store = get_chroma_client()
vector_store.add_documents(docs)
The above code will store the docs in-memory.
If you want to store the docs in hard disk instead of in-memory, run vector_store.persist() at the end. Make sure you have defined persist_directory parameter when defining Chroma client for it.
Note - Having trouble with the assessment engine? Follow the steps listed here
Loading comments...