Charlie Arehart talks about “The Impact Of Unexpected Load and How To Counter It” in this episode of ColdFusion Alive podcast with host Michaela Light. He was one of the speakers at the CF.Objective Conference. In this session, veteran server troubleshooter Charlie Arehart will guide a more detailed review of the issues above, including how to identify such traffic, more on these specific impacts, and most important identifying the solutions along with their pros and cons. He has helped shops achieve dramatic reductions in impact from such automated requests, resulting in greater server stability and performance.
Show notes
Are Spiders Eating Your Servers? The Impact Of Their Unexpected Load and How To Counter It
- Where does unexpected load come from?
- How to identify spiders eating your servers
- DOS attacks, script kiddies
- Malformed requests
- What Google Analytics misses in load
- Load balancer testing
- Use a test page rather than probing your homepage every 5 seconds
- What about robots.txt? Doesn’t that block bots?
- SaaS Tools that search for “bad” traffic flooding your server and don’t let it into your server
- Older CFer, older resources
- Why are you proud to use CF?
- WWIT for you to make CF more alive this year?
- What are you looking forward to at CFObjective?
For years I’ve watched people try to tame “server problems” with a focus on their code, their SQL, the jvm, and so on. Yet often it turns out that the root cause is actually unexpected load. And that load may be from things you never expected (automated), at volumes you never expected. I’ve found folks with as much as 80% of their web traffic to be such unexpected automated traffic! Worse, there are characteristics of such automated visits that may actually have MORE IMPACT than “real users”: for instance, did you know they create a new session–and run session startup code–for each page they visit?!
The good news is there are solutions to better manage (or simply block) such automated requests which may already exist in your environment, and tools you may consider (some free, some commercial) which can be easily implemented. There are even SAAS solutions that could help alleviate such problems with just a single tiny change in your environment! You may also want to consider some admin configuration options related to sessions and/or client variables, as well as reconsider some coding choices in your session startup code.
Mentioned in this episode
- Web spiders
- FusionReactor
- User agent header faking
- IP address spoofing
- On session start
- Load Testing
- Robots.txt
- CF411.com
- Cloudflare
- Cloud/SAAS Firewall-level Application Firewalls tools list
- CFML Job Resources
- Ortus Solutions
- CF Alive episode on What's New In CF 10, 11, And 2016 That You May Have Missed?
- Mike Brunt
- Scary DBA
- Pass conference
- Database fundamentals
- CF Slack channel
- FB programmers group
- TeraTech blog on CFML online groups
- Mary Jo Sminkey CFObj Advanced Error Handling Strategies ColdFusion Alive podcast episode
- Gert Franz CFObj Debugging ColdFusion Alive podcast episode
- CFML Resource Sites
- Adobe CF blog blogs.coldfusion.com
- 2000 sales of CF per quarter
- CF Summit
- CF Meetup
- CFObjective Twitter hashtag
- Things you can do in DC
Listen to the Audio
Bio
Charlie Arehart
A veteran server troubleshooter who’s worked in enterprise IT for more than three decades, Charlie Arehart (@carehart) is a longtime community contributor who as an independent consultant provides short-term, remote, on-demand troubleshooting/tuning assistance for organizations of all sizes and experience levels (carehart.org/consulting).
Links
Interview transcript
Michael: Welcome back to the show. I'm here with Charlie Arehart and he's a veteran ColdFusion troubleshooter. He has been doing I.T. stuff for so long. We don't even want to talk about that until the second half of the show when we will talk about how long we've been doing things and the benefits of that.
But first of all, we're going to talk about unexpected load and how you can counter it and how they may be spiders or other things you didn't even know about. In fact, could you even get to expecting unexpected load? So, lots of crazy stuff there. Charlie’s going to be talking about that at cf.Objective which is coming up soon. So, we'll talk a little bit about that.
So welcome back to the show Charlie.
Charlie: Thank you for having me Michael. Good to see you.
Michael: Yeah, yeah good to see you too coming to us from beautiful downtown Kentucky.
Charlie: Not downtown, beautiful rural out.
Michael: rural?
Charlie: Yes, you're out in the sticks. You escape from the sick city town that's got fifteen hundred people in it.
Michael: Goodness me!
Charlie: That was from a city that's got you know, fifty thousand people in it. I love it.
Michael: You’re quite a way away from dizziness.
Charlie: And it’s beautiful here.
Michael: You know your land used to have…
Charlie: Right the land and then D.C. grown up in D.C.
Michael: yeah
Charlie: Forty's and then moving to Atlanta from my forties and now here and working from here. And that's why we came here because with the work that I do, it's all remote. I can connect to people and log you know, as long as I got a decent internet connection. I can work from here and I'm looking out at the beautiful vista of trees and sometimes turkeys and deer and all kinds of fun stuff.
Michael: That’s amazing, it's incredible what technology has allowed people to do and I’m sure some of the listeners are working with remote as well.
Read more
And to continue learning how to make your ColdFusion apps more modern and alive, I encourage you to download our free ColdFusion Alive Best Practices Checklist.
Because… perhaps you are responsible for a mission-critical or revenue-generating CF application that you don’t trust 100%, where implementing new features is a painful ad-hoc process with slow turnaround even for simple requests.
What if you have no contingency plan for a sudden developer departure or a server outage? Perhaps every time a new freelancer works on your site, something breaks. Or your application availability, security, and reliability are poor.
And if you are depending on ColdFusion for your job, then you can’t afford to let your CF development methods die on the vine.
You’re making a high-stakes bet that everything is going to be OK using the same old app creation ways in that one language — forever.
All it would take is for your fellow CF developer to quit or for your CIO to decide to leave the (falsely) perceived sinking ship of CFML and you could lose everything—your project, your hard-won CF skills, and possibly even your job.
Luckily, there are a number of simple, logical steps you can take now to protect yourself from these obvious risks.
No Brainer ColdFusion Best Practices to Ensure You Thrive No Matter What Happens Next
ColdFusion Alive Best Practices Checklist
Modern ColdFusion development best practices that reduce stress, inefficiency, project lifecycle costs while simultaneously increasing project velocity and innovation.
√ Easily create a consistent server architecture across development, testing, and production
√ A modern test environment to prevent bugs from spreading
√ Automated continuous integration tools that work well with CF
√ A portable development environment baked into your codebase… for free!
Learn about these and many more strategies in our free ColdFusion Alive Best Practices Checklist.