Database activity monitoring
Encyclopedia
Database activity monitoring (DAM) is a database security
technology for monitoring and analyzing database activity that operates independently of the database management system
(DBMS) and does not rely on any form of native (DBMS-resident) auditing or native logs such as trace or transaction logs. DAM is typically performed continuously and in real-time.
Database activity monitoring and prevention (DAMP) is an extension to DAM that goes beyond monitoring and alerting to also block unauthorized activities.
DAM helps businesses address regulatory compliance
mandates like the Payment Card Industry Data Security Standard (PCI DSS), the Health Insurance Portability and Accountability Act
(HIPAA), the Sarbanes-Oxley Act
(SOX), U.S. government regulations such as NIST 800-53, and EU regulations.
DAM is also an important technology for protecting sensitive databases from external attacks by cybercriminals. According to the 2009 Verizon Business’ Data Breach Investigations Report—based on data analyzed from Verizon Business’ caseload of 90 confirmed breaches involving 285 million compromised records during 2008—75 percent of all breached records came from compromised database servers.
According to Gartner
, “DAM provides privileged user and application access monitoring that is independent of native database logging and audit functions. It can function as a compensating control for privileged user separation-of-duties issues by monitoring administrator activity. The technology also improves database security by detecting unusual database read and update activity from the application layer. Database event aggregation, correlation and reporting provide a database audit capability without the need to enable native database audit functions (which become resource-intensive as the level of auditing is increased).”
According to a survey by the Independent Oracle User Group (IOUG), “Most organizations do not have mechanisms in place to prevent database administrators and other privileged database users from reading or tampering with sensitive information in financial, HR, or other business applications. Most are still unable to even detect such breaches or incidents.”
Forrester
refers to this category as “database auditing and real-time protection”.
, developers, help desk
, and outsourced personnel – who typically have unfettered access to corporate databases – is essential for protecting against both external and internal threats. Privileged user monitoring includes auditing all activities and transactions; identifying anomalous activities (such as viewing sensitive data, or creating new accounts with superuser privileges); and reconciling observed activities (such as adding or deleting tables) with authorized change requests.
Since most organizations are already protected at the perimeter level, indeed a major concern lies with the need to monitor and protect from privileged users. There is a high correlation therefore between Database Security
and the need to protect from the insider threat. This is a complex task as most privileged users are capable of using sophisticated techniques to attack the database - stored procedures, triggers, views and obfuscated traffic - attacks that may be difficult to detect using traditional methods.
In addition, since targeted attacks frequently result in attackers gaining privileged user credentials, monitoring of privileged activities is also an effective way to identify compromised systems.
As a result, auditors are now demanding monitoring of privileged users for security best practices as well as a wide range of regulations. Privileged user monitoring helps ensure:
• Data privacy
, so that only authorized applications and users are viewing sensitive data.
• Data governance
, so that critical database structures and values are not being changed outside of corporate change control procedures.
Application Activity Monitoring: The primary purpose of application activity monitoring is to provide a greater level of end-user accountability and detect fraud
(and other abuses of legitimate access) that occurs via enterprise applications, rather than via direct access to the database.
Multi-tier enterprise applications such as Oracle
EBS, PeopleSoft
, JD Edwards, SAP
, Siebel Systems
, Business Intelligence, and custom applications built on standard middle-tier servers such as IBM WebSphere
and Oracle WebLogic Server
mask the identity of end-users at the database transaction level. This is done with an optimization mechanism known as “connection pooling.” Using pooled connections, the application aggregates all user traffic within a few database connections that are identified only by a generic service account name. Application activity monitoring allows organizations to associate specific database transactions with particular application end-users, in order to identify unauthorized or suspicious activities.
End-user accountability is often required for data governance
requirements such as the Sarbanes–Oxley Act. New auditor guidance from the Public Company Accounting Oversight Board
for SOX
compliance has also increased the emphasis on anti-fraud controls.
Cyberattack Protection: SQL injection
is a type of attack used to exploit bad coding practices in applications that use relational databases. The attacker uses the application to send a SQL
statement that is composed from an application statement concatenated with an additional statement that the attacker introduces.
Many application developers compose SQL
statements by concatenating strings and do not use prepared statement; in this case the application is susceptible to a SQL injection
attack. The technique transforms an application SQL statement from an innocent SQL call to a malicious call that can cause unauthorized access, deletion of data, or theft of information.
One way that DAM can prevent SQL injection
is by monitoring the application activity, generating a baseline of “normal behavior”, and identifying an attack based on a divergence from normal SQL
structures and normal sequences. Alternative approaches monitor the memory of the database, where both the database execution plan and the context of the SQL statements are visible, and based on policy can provide granular protection at the object level.
More advanced DAM functions include:
Some enterprises are also seeking other functions, including:
level, or at the level of the database libraries.
If there is unencrypted network traffic, then packet sniffing can be used. The advantage is that no processing is done on the host, however the main disadvantage is that both local traffic and sophisticated intra-database attacks will not be detected. To capture local access some network based vendors deploy a probe that runs on the host. This probe intercepts all local access and can also intercept all networked access in case you do not want to use network gear or in case the database communications are encrypted. However, since the agent does not do all the processing — instead it relays the data to the DAM appliance where all the processing occurs — it may impact network performance with all of the local traffic and real-time session termination may be too slow to interrupt unauthorized queries.
Memory-based: Some DAM systems have a light weight sensor that attaches to the protected databases and continuously polls the system global area
(SGA) to collect SQL
statements as they are being performed. A similar architecture was previously used by performance optimization products that also used the SGA and other shared data structures.
In the latest versions of this technology a light weight sensor runs on the host and attaches to the process at the OS
level to inspect private data structures. The advantages of this approach are significant:
• Complete coverage of all database transactions — the sensor covers traffic coming from the network, from the host, as well as from back-doors (stored procedures, triggers, views)
• A solution that is agnostic to most IT infrastructure variables - no need to re-architect the network, to open span ports or to worry about key management if the network is encrypted, and this model can also be used to protect databases deployed in virtualized environments or in the cloud
Log-based: Some DAM systems analyze and extract the information from the transaction logs (e.g., the redo logs). These systems use the fact that much of the data is stored within the redo logs and they scrape these logs. Unfortunately, not all of the information that is required is in the redo logs. For example, SELECT statements are not and so these systems will augment the data that they gather from the redo logs with data that they collect from the native audit trails as shown in Figure 3. These systems are a hybrid between a true DAM system (that is fully independent from the DBMS) and a SIEM which relies on data generated by the database. These architectures usually imply more overhead on the database server.
Database security
Database security concerns the use of a broad range of information security controls to protect databases against compromises of their confidentiality, integrity and availability...
technology for monitoring and analyzing database activity that operates independently of the database management system
Database management system
A database management system is a software package with computer programs that control the creation, maintenance, and use of a database. It allows organizations to conveniently develop databases for various applications by database administrators and other specialists. A database is an integrated...
(DBMS) and does not rely on any form of native (DBMS-resident) auditing or native logs such as trace or transaction logs. DAM is typically performed continuously and in real-time.
Database activity monitoring and prevention (DAMP) is an extension to DAM that goes beyond monitoring and alerting to also block unauthorized activities.
DAM helps businesses address regulatory compliance
Regulatory compliance
In general, compliance means conforming to a rule, such as a specification, policy, standard or law. Regulatory compliance describes the goal that corporations or public agencies aspire to in their efforts to ensure that personnel are aware of and take steps to comply with relevant laws and...
mandates like the Payment Card Industry Data Security Standard (PCI DSS), the Health Insurance Portability and Accountability Act
Health Insurance Portability and Accountability Act
The Health Insurance Portability and Accountability Act of 1996 was enacted by the U.S. Congress and signed by President Bill Clinton in 1996. It was originally sponsored by Sen. Edward Kennedy and Sen. Nancy Kassebaum . Title I of HIPAA protects health insurance coverage for workers and their...
(HIPAA), the Sarbanes-Oxley Act
Sarbanes-Oxley Act
The Sarbanes–Oxley Act of 2002 , also known as the 'Public Company Accounting Reform and Investor Protection Act' and 'Corporate and Auditing Accountability and Responsibility Act' and commonly called Sarbanes–Oxley, Sarbox or SOX, is a United States federal law enacted on July 30, 2002, which...
(SOX), U.S. government regulations such as NIST 800-53, and EU regulations.
DAM is also an important technology for protecting sensitive databases from external attacks by cybercriminals. According to the 2009 Verizon Business’ Data Breach Investigations Report—based on data analyzed from Verizon Business’ caseload of 90 confirmed breaches involving 285 million compromised records during 2008—75 percent of all breached records came from compromised database servers.
According to Gartner
Gartner
Gartner, Inc. is an information technology research and advisory firm headquartered in Stamford, Connecticut, United States. It was known as GartnerGroup until 2001....
, “DAM provides privileged user and application access monitoring that is independent of native database logging and audit functions. It can function as a compensating control for privileged user separation-of-duties issues by monitoring administrator activity. The technology also improves database security by detecting unusual database read and update activity from the application layer. Database event aggregation, correlation and reporting provide a database audit capability without the need to enable native database audit functions (which become resource-intensive as the level of auditing is increased).”
According to a survey by the Independent Oracle User Group (IOUG), “Most organizations do not have mechanisms in place to prevent database administrators and other privileged database users from reading or tampering with sensitive information in financial, HR, or other business applications. Most are still unable to even detect such breaches or incidents.”
Forrester
Forrester
Forrester may refer to:* Forrester Research, a market research company* Forrester RFC, a Scottish Rugby Union football club* Forrester , people with the surname Forrester-See also:...
refers to this category as “database auditing and real-time protection”.
Common use cases for DAM
Privileged User Monitoring: Monitoring privileged users (or superusers), such as database administrators (DBAs), systems administrators (or sysadmins)System administrator
A system administrator, IT systems administrator, systems administrator, or sysadmin is a person employed to maintain and operate a computer system and/or network...
, developers, help desk
Help desk
A help desk is an information and assistance resource that troubleshoots problems with computers or similar products. Corporations often provide help desk support to their customers via a toll-free number, website and e-mail. There are also in-house help desks geared toward providing the same kind...
, and outsourced personnel – who typically have unfettered access to corporate databases – is essential for protecting against both external and internal threats. Privileged user monitoring includes auditing all activities and transactions; identifying anomalous activities (such as viewing sensitive data, or creating new accounts with superuser privileges); and reconciling observed activities (such as adding or deleting tables) with authorized change requests.
Since most organizations are already protected at the perimeter level, indeed a major concern lies with the need to monitor and protect from privileged users. There is a high correlation therefore between Database Security
Database security
Database security concerns the use of a broad range of information security controls to protect databases against compromises of their confidentiality, integrity and availability...
and the need to protect from the insider threat. This is a complex task as most privileged users are capable of using sophisticated techniques to attack the database - stored procedures, triggers, views and obfuscated traffic - attacks that may be difficult to detect using traditional methods.
In addition, since targeted attacks frequently result in attackers gaining privileged user credentials, monitoring of privileged activities is also an effective way to identify compromised systems.
As a result, auditors are now demanding monitoring of privileged users for security best practices as well as a wide range of regulations. Privileged user monitoring helps ensure:
• Data privacy
Data privacy
Information privacy, or data privacy is the relationship between collection and dissemination of data, technology, the public expectation of privacy, and the legal and political issues surrounding them....
, so that only authorized applications and users are viewing sensitive data.
• Data governance
Data governance
Data governance is an emerging discipline with an evolving definition. The discipline embodies a convergence of data quality, data management, data policies, business process management, and risk management surrounding the handling of data in an organization...
, so that critical database structures and values are not being changed outside of corporate change control procedures.
Application Activity Monitoring: The primary purpose of application activity monitoring is to provide a greater level of end-user accountability and detect fraud
Fraud
In criminal law, a fraud is an intentional deception made for personal gain or to damage another individual; the related adjective is fraudulent. The specific legal definition varies by legal jurisdiction. Fraud is a crime, and also a civil law violation...
(and other abuses of legitimate access) that occurs via enterprise applications, rather than via direct access to the database.
Multi-tier enterprise applications such as Oracle
Oracle Corporation
Oracle Corporation is an American multinational computer technology corporation that specializes in developing and marketing hardware systems and enterprise software products – particularly database management systems...
EBS, PeopleSoft
PeopleSoft
PeopleSoft, Inc. was a company that provided Human Resource Management Systems , Financial Management Solutions , Supply Chain and customer relationship management software, as well as software solutions for manufacturing, enterprise performance management, and student administration to large...
, JD Edwards, SAP
SAP ERP
The SAP ERP application is an integrated enterprise resource planning software manufactured by SAP AG that targets business software requirements of midsize and large organizations in all industries and sectors...
, Siebel Systems
Siebel Systems
Siebel CRM Systems, Inc. was a software company principally engaged in the design, development, marketing, and support of customer relationship management applications. The company was founded by Thomas Siebel in 1993. At first known mainly for its sales force automation products, the company...
, Business Intelligence, and custom applications built on standard middle-tier servers such as IBM WebSphere
IBM WebSphere
IBM WebSphere refers to a brand of computer software products in the genre of enterprise software known as "application and integration middleware". These software products are used by end-users to create applications and integrate applications with other applications...
and Oracle WebLogic Server
Oracle Weblogic Server
Owned by Oracle Corporation, Oracle WebLogic consists of a Java EE platform product-family that includes:* a Java EE application server, WebLogic Application Server* an enterprise portal, WebLogic Portal* an Enterprise Application Integration platform...
mask the identity of end-users at the database transaction level. This is done with an optimization mechanism known as “connection pooling.” Using pooled connections, the application aggregates all user traffic within a few database connections that are identified only by a generic service account name. Application activity monitoring allows organizations to associate specific database transactions with particular application end-users, in order to identify unauthorized or suspicious activities.
End-user accountability is often required for data governance
Data governance
Data governance is an emerging discipline with an evolving definition. The discipline embodies a convergence of data quality, data management, data policies, business process management, and risk management surrounding the handling of data in an organization...
requirements such as the Sarbanes–Oxley Act. New auditor guidance from the Public Company Accounting Oversight Board
Public Company Accounting Oversight Board
The Public Company Accounting Oversight Board is a private-sector, non-profit corporation created by the Sarbanes–Oxley Act, a 2002 United States federal law, to oversee the auditors of public companies. Its stated purpose is to 'protect the interests of investors and further the public interest...
for SOX
SoX
Sound eXchange, abbreviated SoX, is a free cross-platform digital audio editor, licensed under the GNU General Public License, and distributed by Chris Bagwell through SourceForge.net...
compliance has also increased the emphasis on anti-fraud controls.
Cyberattack Protection: SQL injection
SQL injection
A SQL injection is often used to attack the security of a website by inputting SQL statements in a web form to get a badly designed website in order to dump the database content to the attacker. SQL injection is a code injection technique that exploits a security vulnerability in a website's software...
is a type of attack used to exploit bad coding practices in applications that use relational databases. The attacker uses the application to send a SQL
SQL
SQL is a programming language designed for managing data in relational database management systems ....
statement that is composed from an application statement concatenated with an additional statement that the attacker introduces.
Many application developers compose SQL
SQL
SQL is a programming language designed for managing data in relational database management systems ....
statements by concatenating strings and do not use prepared statement; in this case the application is susceptible to a SQL injection
SQL injection
A SQL injection is often used to attack the security of a website by inputting SQL statements in a web form to get a badly designed website in order to dump the database content to the attacker. SQL injection is a code injection technique that exploits a security vulnerability in a website's software...
attack. The technique transforms an application SQL statement from an innocent SQL call to a malicious call that can cause unauthorized access, deletion of data, or theft of information.
One way that DAM can prevent SQL injection
SQL injection
A SQL injection is often used to attack the security of a website by inputting SQL statements in a web form to get a badly designed website in order to dump the database content to the attacker. SQL injection is a code injection technique that exploits a security vulnerability in a website's software...
is by monitoring the application activity, generating a baseline of “normal behavior”, and identifying an attack based on a divergence from normal SQL
SQL
SQL is a programming language designed for managing data in relational database management systems ....
structures and normal sequences. Alternative approaches monitor the memory of the database, where both the database execution plan and the context of the SQL statements are visible, and based on policy can provide granular protection at the object level.
Core features of DAM
As defined by Gartner, “DAM tools use several data collection mechanisms (such as server-based agent software and in-line or out-of-band network collectors), aggregate the data in a central location for analysis, and report based on behaviors that violate the security policies and/or signatures or indicate behavioral anomalies. DAM demand is driven primarily by the need for privileged user monitoring to address compliance-related audit findings, and by threat-management requirements to monitor database access. Enterprise DAM requirements are beginning to broaden, extending beyond basic functions, such as the capability to detect malicious activity or inappropriate or unapproved database administrator (DBA) access.”More advanced DAM functions include:
- The ability to monitor intra-database attacks and back-doors in real time (such as stored procedures, triggers, views, etc.)
- A solution which is agnostic to most IT infrastructure variables - such as encryption or network topology
- Blocking and prevention, without being in-line to the transactions
- Active discovery of at-risk data
- Improved visibility into application traffic
- The ability to offer database activity monitoring in virtualized environments, or even in the cloud, where there is no well-defined or consistent network topology
Some enterprises are also seeking other functions, including:
- Configuration auditing to comply with audits required by the U.S. Sarbanes-Oxley Act
- DLP capabilities that address security concerns, as well as the data identification and protection requirements of the Payment Card Industry (PCI) and other data-centric regulatory frameworks
- Database user rights attestation reporting, required by a broad range of regulations
- The ability to offer database activity monitoring in virtualized environments, or even in the cloud, where there is no well-defined or consistent network topology
- Better integration with vulnerability scanning products
Common DAM architectures
Interception-based: Most modern DAM systems collect what the database is doing by being able to “see” the communications between the database client and the database server. What DAM systems do is find places where they can view the communication stream and get the requests and responses without requiring participation from the database. The interception itself can be done at multiple points such as the database memory (e.g. the SGA), at the network (using a network TAP or a SPAN port if the communication is not encrypted), at the operating systemOperating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
level, or at the level of the database libraries.
If there is unencrypted network traffic, then packet sniffing can be used. The advantage is that no processing is done on the host, however the main disadvantage is that both local traffic and sophisticated intra-database attacks will not be detected. To capture local access some network based vendors deploy a probe that runs on the host. This probe intercepts all local access and can also intercept all networked access in case you do not want to use network gear or in case the database communications are encrypted. However, since the agent does not do all the processing — instead it relays the data to the DAM appliance where all the processing occurs — it may impact network performance with all of the local traffic and real-time session termination may be too slow to interrupt unauthorized queries.
Memory-based: Some DAM systems have a light weight sensor that attaches to the protected databases and continuously polls the system global area
System Global Area
In the database management systems developed by the Oracle Corporation, the System Global Area forms the part of the RAM shared by all the processes belonging to a single Oracle database instance...
(SGA) to collect SQL
SQL
SQL is a programming language designed for managing data in relational database management systems ....
statements as they are being performed. A similar architecture was previously used by performance optimization products that also used the SGA and other shared data structures.
In the latest versions of this technology a light weight sensor runs on the host and attaches to the process at the OS
OS
OS may refer to:* O.S. Old Stonyhurst, an old boy of the ancient Jesuit public school, Stonyhurst College* O.S. Engines, a Japanese manufacturer of model aircraft engines* Ocean Science, an Oceanographic Journal published by the European Geosciences Union....
level to inspect private data structures. The advantages of this approach are significant:
• Complete coverage of all database transactions — the sensor covers traffic coming from the network, from the host, as well as from back-doors (stored procedures, triggers, views)
• A solution that is agnostic to most IT infrastructure variables - no need to re-architect the network, to open span ports or to worry about key management if the network is encrypted, and this model can also be used to protect databases deployed in virtualized environments or in the cloud
Log-based: Some DAM systems analyze and extract the information from the transaction logs (e.g., the redo logs). These systems use the fact that much of the data is stored within the redo logs and they scrape these logs. Unfortunately, not all of the information that is required is in the redo logs. For example, SELECT statements are not and so these systems will augment the data that they gather from the redo logs with data that they collect from the native audit trails as shown in Figure 3. These systems are a hybrid between a true DAM system (that is fully independent from the DBMS) and a SIEM which relies on data generated by the database. These architectures usually imply more overhead on the database server.
Additional reading
- The Future of Database Activity Monitoring, Jeffrey Wheatman, Garntner
- http://www.darkreading.com/database_security/index.jhtml
- http://www.scmagazineus.com/Database-security-Protecting-the-crown-jewels/article/126934/
- Your Enterprise Database Security Strategy, Noel Yuhanna, Forrester
- http://www.enterprisestrategygroup.com/Login.asp?frompage=BuyInsight.asp&ReportID=1252
- http://www.enterprisestrategygroup.com/Login.asp?frompage=BuyInsight.asp&ReportID=1251
- http://www.burtongroup.com/research/PublicDocument.aspx?cid=1536
- http://www.gartner.com/DisplayDocument?ref=g_rss&id=873513
- Dark Reading - Tech Insight: Database Activity Monitoring
- http://www.appsecinc.com/media/ESG-2010/ESG-Research-Summary-AppSecInc-Database-Security-December-2009.pdf
- http://blog.imperva.com/2011/08/insider-threats-quantifying-the-problem.html
- http://blog.imperva.com/2011/08/outing-the-insiders.html