Just to caveat this - I think agencies definitely have their place for certain companies who are the right fit when it comes to SEO.
I work with agencies and have in the past - and most of them are full of great people - I may even do a blog post addressing why you should choose an agency over a freelance SEO - but this one addresses why you should choose the little guy!
1. One dedicated point of contactΒ
You don't get passed from pillar to post between people - if you wanna check your organic traffic - there will be one person (in my case me!)Β
Agencies can cause confusion for clients with their being a head of SEO, an account manager and a junior SEO - and they all have different purposes.Β
Then when one is on holiday - it becomes even more confusing!Β
2. No generalising, just specialising
You don't deal with people carrying multiple hats - it's just a guy who does SEO. They've been geeking over it for years and would definitely choose it as their specialist topic for a pub quiz.
They've been concentrated on one topic for so long (SEO) that they have to keep expanding their knowledge naturally, all the while keeping up to date with any algorithm or SEO tool changes. No jack of all trades here!

3. Consistency you can count onΒ
Agency staff can change frequently - especially with larger agencies. Junior staff leave to get better pay and senior staff get headhunted. Freelance SEOs are in control of their clients and earnings.
This ties in to the first point around keeping one point of contact. Experienced freelance SEOs have had to develop processes and be iterative over time. Whereas established agencies can have quite stale processes or newer agencies haven't established one yet.
4. Quicker response timesΒ
When a Google algorithm update hits your site out of the blue - you can call/text/email/send a carrier pigeon and you can expect a fast response (but do try and avoid a 4am panic text please)
I tend to use whatsapp or slack with clients which are both installed on my phone - so I can respond very quickly and make any website changes if my laptop is with me.

5. Freelance SEOs have fewer clients to manage
This means they have more dedicated time to learn about your business, industry, organic competitors and everything really. Agency account managers tend to have tons of people and clients to manage and are spread thinly (trust me I've been one!)
It's very difficult for junior SEO account managers to say they are at full capacity to their manager - whereas freelance SEO consultants are used to having strong boundaries with new prospects if they're not a good fit.
6. Greater commitment to positive results
Freelance SEOs (like me) have a strong reputation to upload. I live and die by referrals and good solid reputations - agency staff can sometimes go under the radar.Β
Reviews - whether they are on google or at a third party website - are an independent SEO specialists bloodline. Agencies tend to have hundreds of reviews so one bad one won't hurt them anywhere near as much.
7. You're paying for expertise rather than overheadsΒ
With a freelance SEO you're paying for their knowledge, experience and software. With agencies you're paying for huge office costs, employees, sales teams and more.Β