Hello @wanghan5! I'm here to help you with any bugs, questions, or contributions. Let's work together to solve this issue.

I found a similar open issue related to performance problems in the Dify codebase. The suggested solution involves optimizing the IndexingRunner().run() method, particularly the _update_segment_keywords() function, by:

1. Extracting keywords during segment insertion.
2. Implementing batch processing to reduce database operation overhead.
3. Utilizing multi-threading or multi-processing to handle multiple segments concurrently.

Additionally, it suggests modifying the DatasetDocumentStore.add_documents method to extract keywords during document insertion, ensuring keywords are inserted into document_segments rather than documents ^[1].

To continue talking to Dosu, mention @dosu.

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

Documents have already been indexed. The performance problem happens when trying to retrieve them.

I found some issues regarding wrong index and missuage of properties.

There is knowledge base rate limit in could version. https://docs.dify.ai/guides/knowledge-base/knowledge-request-rate-limit

I would like to sumbit a pull request later. Hope it helps.