Been Taken In Or Distracted By Silly (And Irrelevant) Assertions About The Legal Status Of The IRS?

Don't be. 

They're nonsense (and so is most everything else usually asserted along with them, regarding the IMF, the UN, the Queen of England etc.--those who have seen this stuff will know what I'm talking about...)

As is always true in the law, context dictates meaning:

Title 28 § 451. Definitions

As used in this title

The term “agency” includes any department, independent establishment, commission, administration, authority, board or bureau of the United States or any corporation in which the United States has a proprietary interest, unless the context shows that such term was intended to be used in a more limited sense. (emphasis added)

That is, an agency is an agency, but if a particular statute has a custom definition of the term "agency" for purposes of its own special and limited context, not everything which generically IS an agency may qualify as meeting that special, limited-application definition for purposes of that statute.

The letter excerpted above is a response to a question about a particular judicial contest, Diversified Metal Products Inc. v. T-Bow Company Trust, et al, in which the Dept. of Justice testified that the IRS is not an "agency of the United States Government".  This declaration has been widely bruited about as supporting the absurd notion that the IRS doesn't really exist, is only a part of the Puerto Rican bureaucracy, or similar nonsense.

However, all that was actually meant by that testimony was that within the particular context of this case the IRS did not qualify as an "agency"-- due to the nature of the issue being litigated and that of the particular statutory authority being litigated (which contained a standard "for purposes of this section" qualifier, along with a custom "agency of the United States means XXXX" definition).

In most other statutory contexts, though, it does; and the IRS is, in fact, a bureau of the Department of Treasury as AUSA Derbidge observes.

See http://www.law.cornell.edu/uscode/html/uscode31/usc_sec_31_00000321----000-.html for general language related to the statutory authority of the Secretary of the Treasury to authorize internal revenue activities and offices, and for more particular relevant provisions, see http://www4.law.cornell.edu/uscode/html/uscode26/usc_sec_26_00007801----000-.html and http://www4.law.cornell.edu/uscode/html/uscode26/usc_sec_26_00007803----000-.html.

(And frankly, think about it.  How credible is the proposition that Congress is incapable of providing the Department of Treasury, or the United States governmental structure generally, with an accounting and collections agency such as the IRS?  After all, that IS what this ridiculous notion suggests in questioning the existence or status of everyone's least-favorite agency...)

Click here for an excellent history of the agency up until the mid 1940s.

Also, see the following. The portion chosen for underlining suggests that the person who secured it didn't really read what followed, but you should...