Not if browser vendors make sure to use an access key activation mechanism that doesn't conflict with known access keys in AT. Like Opera's Shift+Esc. And not if AT vendors make sure not to use access keys that may conflict with the access key activation mechanism in known browsers.
No, you're not understanding the concept of accessibility properly. Inaccessible means that you cannot access the content, or critical functions. But access keys aren't critical functions; they are just progressive enhancement for those who have something to gain from them.
Some onus is also on the user. There is numerous browsers to choose from – most of them free of charge and available cross-platform – and they employ different access key activation mechanisms: Alt, Shift+Alt, Ctrl, Shift+Esc, ... So the likelihood that you cannot find a single browser that doesn't conflict with your AT should be quite small.
And even if that happens, the result is nothing worse than your missing out on a few short cuts on a handful of web sites.
Compared to the accessibility problems caused by abusing markup for presentational purposes, missing text equivalents, improper heading order, poor contrast, reliance on client-side scripting or plug-ins, missing background colour specification on elements with white text on a background image, etc., etc., etc., I'd say this is a minor thing.
I'm not advocating the use of access keys, but the reason isn't that there may be conflicts in some combinations of UA+AT, but that they are virtually useless anyway. As far as I know, Opera is the only browser that can even show you what access keys are available on a particular web page. And most sites that use access keys don't indicate what they are, except maybe on a separate page somewhere.