When working the Chat APIs, one common parameter is stream. This dictates how the API responds to your query.

stream=false

When the stream is set to False, the API makes a request, and the response is in one (typically JSON) file. In a chat, this means that the endpoint waits for the LLM to generate the entire response and return this at once.

Works like a traditional API

Slow - The LLM can take 15=30 seconds to craft the entire response

[
  {
    "choices": [
      {
        "delta": {
          "content": "The Great Pyramids of Egypt, particularly the Pyramids of Giza, were constructed as monumental tombs for the pharaohs and are located in Egypt due to the historical and cultural significance of the region. Here are some key reasons why they are situated there:\n\n1. **Historical and Cultural Significance**: Ancient Egypt was one of the world's earliest and most influential civilizations. The Nile River Valley, where the pyramids are located, was the heart of this civilization, providing fertile land and resources that supported a thriving society.\n\n2. **Religious Beliefs**: The ancient Egyptians believed in an afterlife and constructed elaborate tombs to ensure the pharaohs' safe passage and well-being in the next world. The pyramids were built as part of this religious tradition, serving as both tombs and monuments to the pharaohs' divine status.\n\n3. **Geographical Location**: The Giza Plateau, where the Great Pyramids are located, provided a stable and elevated site suitable for large-scale construction. The proximity to the Nile River facilitated the transportation of materials, such as limestone and granite, used in the construction of the pyramids.\n\n4. **Political Power**: The construction of the pyramids demonstrated the pharaohs' power and resources. Building such monumental structures required a highly organized society with advanced knowledge in engineering, architecture, and labor management.\n\n5. **Legacy and Continuity**: The pyramids were part of a long tradition of monumental architecture in ancient Egypt. They were built during the Old Kingdom period, around 2580–2560 BCE, and followed earlier developments in tomb construction, such as mastabas and step pyramids.\n\nIn summary, the Great Pyramids are in Egypt because of the region's historical, cultural, and religious significance, as well as its geographical suitability for such grand construction projects. They stand as enduring symbols of the ancient Egyptian civilization's achievements and beliefs."
        },
        "finish_reason": "stop"
      }
    ]
  }
]

stream=true

With this parameter, multiple JSON objects are returned to the server - often with one word (or short phrase) in the response. This gives more of a 'feel' of typig, and is the live feedback you experience with Corcel (and other LLM baed tools.

Different API response structure

Faster, text appears naturally

data: {"choices": [{"delta": {"content": "The", "role": "assistant"}}]}

data: {"choices": [{"delta": {"content": " Great P"}}]}

data: {"choices": [{"delta": {"content": "yramids"}}]}

data: {"choices": [{"delta": {"content": " of Egypt"}}]}

data: {"choices": [{"delta": {"content": ", particularly"}}]}

data: {"choices": [{"delta": {"content": " the"}}]}

data: {"choices": [{"delta": {"content": " Pyramids of"}}]}

data: {"choices": [{"delta": {"content": " Giza"}}]}

data: {"choices": [{"delta": {"content": ", were"}}]}

data: {"choices": [{"delta": {"content": " constructed as"}}]}

data: {"choices": [{"delta": {"content": " monumental tomb"}}]}

data: {"choices": [{"delta": {"content": "s"}}]}

data: {"choices": [{"delta": {"content": " for"}}]}

data: {"choices": [{"delta": {"content": " the ph"}}]}

data: {"choices": [{"delta": {"content": "araohs"}}]}

data: {"choices": [{"delta": {"content": " and"}}]}

data: {"choices": [{"delta": {"content": " are"}}]}

data: {"choices": [{"delta": {"content": " located in"}}]}

data: {"choices": [{"delta": {"content": " Egypt due to the"}}]}

data: {"choices": [{"delta": {"content": " historical and"}}]}

data: {"choices": [{"delta": {"content": " cultural significance"}}]}

data: {"choices": [{"delta": {"content": " of the"}}]}

data: {"choices": [{"delta": {"content": " region."}}]}

data: {"choices": [{"delta": {"content": " Here are"}}]}

data: {"choices": [{"delta": {"content": " several reasons"}}]}

data: {"choices": [{"delta": {"content": " why they"}}]}

data: {"choices": [{"delta": {"content": " are situated"}}]}

data: {"choices": [{"delta": {"content": " there:\n\n"}}]}

data: {"choices": [{"delta": {"content": "1."}}]}

data: {"choices": [{"delta": {"content": " **Historical"}}]}

data: {"choices": [{"delta": {"content": " and Cultural"}}]}

data: {"choices": [{"delta": {"content": " Significance"}}]}

data: {"choices": [{"delta": {"content": "**:"}}]}

data: {"choices": [{"delta": {"content": " Ancient Egypt"}}]}

data: {"choices": [{"delta": {"content": " was one"}}]}

data: {"choices": [{"delta": {"content": " of the"}}]}

data: {"choices": [{"delta": {"content": " world's earliest"}}]}

data: {"choices": [{"delta": {"content": " and most"}}]}

data: {"choices": [{"delta": {"content": " influential civilizations"}}]}

data: {"choices": [{"delta": {"content": "."}}]}

data: {"choices": [{"delta": {"content": " The"}}]}

data: {"choices": [{"delta": {"content": " Nile"}}]}

data: {"choices": [{"delta": {"content": " River Valley"}}]}

data: {"choices": [{"delta": {"content": ", where"}}]}

data: {"choices": [{"delta": {"content": " the pyram"}}]}

data: {"choices": [{"delta": {"content": "ids are"}}]}

data: {"choices": [{"delta": {"content": " located,"}}]}

data: {"choices": [{"delta": {"content": " was the"}}]}

data: {"choices": [{"delta": {"content": " heart"}}]}

data: {"choices": [{"delta": {"content": " of"}}]}

data: {"choices": [{"delta": {"content": " this civilization"}}]}

data: {"choices": [{"delta": {"content": ", providing"}}]}

data: {"choices": [{"delta": {"content": " fertile land and resources that supported"}}]}

data: {"choices": [{"delta": {"content": " a thriving"}}]}

data: {"choices": [{"delta": {"content": " society.\n\n"}}]}

data: {"choices": [{"delta": {"content": "2."}}]}

data: {"choices": [{"delta": {"content": " **Religious Bel"}}]}

data: {"choices": [{"delta": {"content": "iefs**"}}]}

data: {"choices": [{"delta": {"content": ": The"}}]}

data: {"choices": [{"delta": {"content": " ancient Egyptians"}}]}

data: {"choices": [{"delta": {"content": " believed in"}}]}

data: {"choices": [{"delta": {"content": " an after"}}]}

data: {"choices": [{"delta": {"content": "life and constructed elaborate tombs to ensure"}}]}

data: {"choices": [{"delta": {"content": " the ph"}}]}

data: {"choices": [{"delta": {"content": "araohs"}}]}

data: {"choices": [{"delta": {"content": "' safe passage"}}]}

data: {"choices": [{"delta": {"content": " and well"}}]}

data: {"choices": [{"delta": {"content": "-being in the next world."}}]}

data: {"choices": [{"delta": {"content": " The pyram"}}]}

data: {"choices": [{"delta": {"content": "ids"}}]}

data: {"choices": [{"delta": {"content": " were"}}]}

data: {"choices": [{"delta": {"content": " part"}}]}

data: {"choices": [{"delta": {"content": " of"}}]}

data: {"choices": [{"delta": {"content": " this religious"}}]}

data: {"choices": [{"delta": {"content": " and cultural"}}]}

data: {"choices": [{"delta": {"content": " practice"}}]}

data: {"choices": [{"delta": {"content": ", serving as"}}]}

data: {"choices": [{"delta": {"content": " monumental burial"}}]}

data: {"choices": [{"delta": {"content": " sites.\n\n"}}]}

data: {"choices": [{"delta": {"content": "3"}}]}

data: {"choices": [{"delta": {"content": "."}}]}

data: {"choices": [{"delta": {"content": " **"}}]}

data: {"choices": [{"delta": {"content": "Ge"}}]}

data: {"choices": [{"delta": {"content": "ographical"}}]}

data: {"choices": [{"delta": {"content": " Consider"}}]}

data: {"choices": [{"delta": {"content": "ations**"}}]}

data: {"choices": [{"delta": {"content": ": The"}}]}

data: {"choices": [{"delta": {"content": " location"}}]}

data: {"choices": [{"delta": {"content": " near"}}]}

data: {"choices": [{"delta": {"content": " the Nile"}}]}

data: {"choices": [{"delta": {"content": " River was strategic"}}]}

data: {"choices": [{"delta": {"content": ". The river provided"}}]}

data: {"choices": [{"delta": {"content": " a means"}}]}

data: {"choices": [{"delta": {"content": " of transporting"}}]}

data: {"choices": [{"delta": {"content": " the massive"}}]}

data: {"choices": [{"delta": {"content": " stones used"}}]}

data: {"choices": [{"delta": {"content": " in the"}}]}

data: {"choices": [{"delta": {"content": " construction of"}}]}

data: {"choices": [{"delta": {"content": " the"}}]}

data: {"choices": [{"delta": {"content": " pyramids."}}]}

data: {"choices": [{"delta": {"content": " Additionally,"}}]}

data: {"choices": [{"delta": {"content": " the G"}}]}

data: {"choices": [{"delta": {"content": "iza Plateau"}}]}

data: {"choices": [{"delta": {"content": " offered a"}}]}

data: {"choices": [{"delta": {"content": " stable and"}}]}

data: {"choices": [{"delta": {"content": " elevated site"}}]}

data: {"choices": [{"delta": {"content": " suitable for"}}]}

data: {"choices": [{"delta": {"content": " such large"}}]}

data: {"choices": [{"delta": {"content": " structures.\n\n"}}]}

data: {"choices": [{"delta": {"content": "4."}}]}

data: {"choices": [{"delta": {"content": " **Political"}}]}

data: {"choices": [{"delta": {"content": " and"}}]}

data: {"choices": [{"delta": {"content": " Economic"}}]}

data: {"choices": [{"delta": {"content": " Power**"}}]}

data: {"choices": [{"delta": {"content": ": The"}}]}

data: {"choices": [{"delta": {"content": " construction of the pyramids demonstrated"}}]}

data: {"choices": [{"delta": {"content": " the ph"}}]}

data: {"choices": [{"delta": {"content": "araohs"}}]}

data: {"choices": [{"delta": {"content": "' power"}}]}

data: {"choices": [{"delta": {"content": " and resources"}}]}

data: {"choices": [{"delta": {"content": ". Building such grand"}}]}

data: {"choices": [{"delta": {"content": " structures required"}}]}

data: {"choices": [{"delta": {"content": " a well"}}]}

data: {"choices": [{"delta": {"content": "-organized"}}]}

data: {"choices": [{"delta": {"content": " state,"}}]}

data: {"choices": [{"delta": {"content": " a large"}}]}

data: {"choices": [{"delta": {"content": " workforce"}}]}

data: {"choices": [{"delta": {"content": ", and"}}]}

data: {"choices": [{"delta": {"content": " significant economic"}}]}

data: {"choices": [{"delta": {"content": " resources,"}}]}

data: {"choices": [{"delta": {"content": " all of"}}]}

data: {"choices": [{"delta": {"content": " which were"}}]}

data: {"choices": [{"delta": {"content": " available in"}}]}

data: {"choices": [{"delta": {"content": " ancient Egypt"}}]}

data: {"choices": [{"delta": {"content": ".\n\n5"}}]}

data: {"choices": [{"delta": {"content": ". **"}}]}

data: {"choices": [{"delta": {"content": "Architectural"}}]}

data: {"choices": [{"delta": {"content": " Innovation**"}}]}

data: {"choices": [{"delta": {"content": ": The"}}]}

data: {"choices": [{"delta": {"content": " Egyptians"}}]}

data: {"choices": [{"delta": {"content": " had"}}]}

data: {"choices": [{"delta": {"content": " developed advanced"}}]}

data: {"choices": [{"delta": {"content": " techniques"}}]}

data: {"choices": [{"delta": {"content": " in engineering"}}]}

data: {"choices": [{"delta": {"content": " and architecture"}}]}

data: {"choices": [{"delta": {"content": ", allowing"}}]}

data: {"choices": [{"delta": {"content": " them to construct"}}]}

data: {"choices": [{"delta": {"content": " these"}}]}

data: {"choices": [{"delta": {"content": " massive structures"}}]}

data: {"choices": [{"delta": {"content": ". The"}}]}

data: {"choices": [{"delta": {"content": " pyramids"}}]}

data: {"choices": [{"delta": {"content": " are a"}}]}

data: {"choices": [{"delta": {"content": " testament to"}}]}

data: {"choices": [{"delta": {"content": " their ingenuity"}}]}

data: {"choices": [{"delta": {"content": " and"}}]}

data: {"choices": [{"delta": {"content": " skill"}}]}

data: {"choices": [{"delta": {"content": ".\n\nIn"}}]}

data: {"choices": [{"delta": {"content": " summary,"}}]}

data: {"choices": [{"delta": {"content": " the Great"}}]}

data: {"choices": [{"delta": {"content": " Pyram"}}]}

data: {"choices": [{"delta": {"content": "ids"}}]}

data: {"choices": [{"delta": {"content": " are in Egypt because"}}]}

data: {"choices": [{"delta": {"content": " of the"}}]}

data: {"choices": [{"delta": {"content": " region's"}}]}

data: {"choices": [{"delta": {"content": " historical,"}}]}

data: {"choices": [{"delta": {"content": " cultural,"}}]}

data: {"choices": [{"delta": {"content": " religious,"}}]}

data: {"choices": [{"delta": {"content": " geographical,"}}]}

data: {"choices": [{"delta": {"content": " and political"}}]}

data: {"choices": [{"delta": {"content": " significance during"}}]}

data: {"choices": [{"delta": {"content": " the time of their construction."}}]}

data: {"choices": [{"delta": {"content": ""}, "finish_reason": "stop"}]}

data: [DONE]