What appears to be a Beretta 92FS with a mounted suppressor is used by a murder suspect at the start of "Internal Audit". Deputy Commissioner Frank Da Silva (Peter Gerety) uses one with a mounted silencer near the end of "All in the Family".
Beretta 92FS Centurion
Near the end of "Paint it Black" (S2E22), Marchef (Henri Lubatti) uses a Beretta 92FS Centurion to execute his badly wounded cousin after Joan suggests him to get him to the nearest hospital.
Defence Distributed Liberator
A Defense Distributed Liberator 3D printed pistol was used by Lawrence Pendry (Rufus Wright) as a murder weapon in "Step Nine" (S02E01). It appears to be incomplete, with the top portions of the sides of the pistol being omitted or removed.
Glock 17
What appears to be a Glock 17 is used by a murder suspect in "Solve For X" (S02E02). Another one appears to be used in a home invasion-style attack against Cheryl Grayson in "An Unnatural Arrangement". A Glock 17 is in the duty holster of an Ulster County Sheriff in "The Marchioness" (S02E07) Detective Marcus Bell (Jon Michael Hill) uses the Glock 17 as his main sidearm as a NYPD plainclothes officer in the season. It is also used by the Emergency Service Unit as its main sidearm.
Glock 19
The Glock 19 is seen throughout the season as the main sidearm for all NYPD officers.
Glock 21
A home intruder threatens Cheryl Gregson (Talia Balsam) in "An Unnatural Arrangement" (S03E06), which she later identifies to the police as a Glock 21.
Heckler & Koch Mark 23
Devon Gaspar (Andrew Howard) uses a suppressed Heckler & Koch Mark 23 as his main weapon throughout the episode "The Diabolical Kind" (S02E12).
M1911A1
What appears to be a M1911A1 with pearl-like grips are found in the shoulder holster of "El Mecanico" (Andrew Samonsky) near the end of "The Marchioness" (S02E07).
SIG-Sauer P226
Some of the mobsters working for de Soto in "Paint it Black" (S02E22) are armed with SIG-Sauer P226 pistols.
Springfield Armory XDM
A Springfield Armory XDM with a 5.25" barrel and adjustable sights is used by Lara Banin (Mia Barron) in "Ancient History" (S02E05).
Walther PPK
A stainless steel Walther PPK was about to be used by Thomas Hauser (Donald Ryan) at the start of "Internal Audit" (S02E11).
A Smith & Wesson Model 681 is used in the revolver wheel, changing from the Taurus Model 689 in the opening sequence. It makes a return since the opening sequence is used again in the second season.
Taurus Model 689
Taurus Model 689 revolvers are used in the opening sequence of Elementary in a revolver wheel. Like the Model 681, it makes a return since the opening sequence is used again in the second season.
Unknown
Submachine Guns
Colt 9mm Submachine Gun
What appears to be a Colt 9mm Submachine Gun without the carry handle is used by an ESU operator near the end of "All in the Family" (S02E13).
Assault Rifles
Colt Law Enforcement Carbine
The Colt Law Enforcement Carbine is used by the ESU in the season as their main assault rifle from the first season.
Colt M4A1
What appear to be Colt M4A1 assault rifles are used by operators of E Squadron aka the Increment at the end of "Paint it Black" (S02E22) when they were able to help Mycroft save him and Watson from being murdered by de Soto's henchmen.
Mark 18 Mod 0
An ESU operator is armed with the Mark 18 Mod 0 near the end of "All in the Family" (S02E13).
Shotguns
Sawed off shotgun
A sawed off shotgun was brought out in an NYPD office at the beginning of "Tremors" (S02E10) by Silas Cole (Zachary Booth) under schizophrenia.
Sniper Rifles
Nemesis Arms Vanquish
El Mechanico (Andrew Samonsky) uses a Nemesis Arms Vanquish in an attempt to assassinate Nigella Mason (Olivia d'Abo) in "The Marchioness" (S2E07).
Grenades
L2A2 Hand Grenade
An L2A2 hand grenade is used by suspended Detective Inspector Gareth Lestrade (Sean Pertwee) of the Metropolitan Police Service as he crashes a funeral in "Step Nine" (S02E01).