alot of really small SLMs seem to be too small to even function properly.
however, there may be some unconventional uses for some of them.

one usecase keyword generation. heres what gemma-3-270m-it-qat-Q4_0 gave me

prompt: keywords similar to “construction”

Construction activities
*   Construction design
*   Construction equipment
*   Construction management
*   Construction materials
*   Construction methodology
*   Construction organization
*   Construction planning
*   Construction process
*   Construction project
*   Construction project details
*   Construction project management
*   Construction project management activities
*   Construction project management assessment
*   Construction project management checklist
*   Construction project management document
*   Construction project management evaluation
*   Construction project management framework
*   Construction project management methodology
*   Construction project management plan
*   Construction project management project
*   Construction project management report
*   Construction project management review
*   Construction project management schedule
*   Construction project management software
*   Construction project management strategy
*   Construction project management system
*   Construction project management tool
*   Construction project planning
*   Construction project scope
*   Construction schedule
*   Construction strategy
*   Construction work
```
                
90 TPS (but got stuck in a repeating loop)          
            
i think theres alot of other uses. but they arent really LLM answer that people usually reach for an LLM.
  • ComradePenguin@lemmy.ml
    link
    fedilink
    English
    arrow-up
    0
    ·
    3 days ago

    I am trying to fully understand this. Can you please elaborate further? Are you thinking about a heuristic based harness?

    • abcdqfr@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      3 days ago

      Not sure about keyword generation exactly as OP intends to apply this, but short simple code that needs to be validated on output seems like the better application. Models are tiny and fast, so mmap and run in parallel and race to what passes validation first. Each agent gets a git branch. Losers generate the anti patterns. Human or cheap tokens integrates the results from corpus back into harness. Next generation.