AA20-183A: Defending Against Malicious Cyber Activity Originating from Tor

This post was originally published on this site

Original release date: July 1, 2020 | Last revised: July 2, 2020

Summary

This advisory uses the MITRE Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK®) and Pre-ATT&CK framework. See the ATT&CK for Enterprise and Pre-ATT&CK frameworks for referenced threat actor techniques.

This advisory—written by the Cybersecurity Security and Infrastructure Security Agency (CISA) with contributions from the Federal Bureau of Investigation (FBI)—highlights risks associated with Tor, along with technical details and recommendations for mitigation. Cyber threat actors can use Tor software and network infrastructure for anonymity and obfuscation purposes to clandestinely conduct malicious cyber operations.[1],[2],[3]

Tor (aka The Onion Router) is software that allows users to browse the web anonymously by encrypting and routing requests through multiple relay layers or nodes. This software is maintained by the Tor Project, a nonprofit organization that provides internet anonymity and anti-censorship tools. While Tor can be used to promote democracy and free, anonymous use of the internet, it also provides an avenue for malicious actors to conceal their activity because identity and point of origin cannot be determined for a Tor software user. Using the Onion Routing Protocol, Tor software obfuscates a user’s identity from anyone seeking to monitor online activity (e.g., nation states, surveillance organizations, information security tools). This is possible because the online activity of someone using Tor software appears to originate from the Internet Protocol (IP) address of a Tor exit node, as opposed to the IP address of the user’s computer.

CISA and the FBI recommend that organizations assess their individual risk of compromise via Tor and take appropriate mitigations to block or closely monitor inbound and outbound traffic from known Tor nodes.

Click here for a PDF version of this report.

Risk Evaluation

Malicious cyber actors use Tor to mask their identity when engaging in malicious cyber activity impacting the confidentiality, integrity, and availability of an organization’s information systems and data. Examples of this activity include performing reconnaissance, penetrating systems, exfiltrating and manipulating data, and taking services offline through denial-of-service attacks and delivery of ransomware payloads. Threat actors have relayed their command and control (C2) server communications—used to control systems infected with malware—through Tor, obscuring the identity (location and ownership) of those servers.

The use of Tor in this context allows threat actors to remain anonymous, making it difficult for network defenders and authorities to perform system recovery and respond to cyberattacks. Organizations that do not take steps to block or monitor Tor traffic are at heightened risk of being targeted and exploited by threat actors hiding their identity and intentions using Tor.

The risk of being the target of malicious activity routed through Tor is unique to each organization. An organization should determine its individual risk by assessing the likelihood that a threat actor will target its systems or data and the probability of the threat actor’s success given current mitigations and controls. This assessment should consider legitimate reasons that non-malicious users may prefer to, or need to, use Tor for accessing the network. Organizations should evaluate their mitigation decisions against threats to their organization from advanced persistent threats (APTs), moderately sophisticated attackers, and low-skilled individual hackers, all of whom have leveraged Tor to carry out reconnaissance and attacks in the past.

Technical Details

Tor obfuscates the source and destination of a web request. This allows users to conceal information about their activities on the web—such as their location and network usage—from the recipients of that traffic, as well as third parties who may conduct network surveillance or traffic analysis. Tor encrypts a user’s traffic and routes the traffic through at least three Tor nodes, or relays, so that the user’s starting IP address and request is masked from network and traffic observers during transit. Once the request reaches its intended destination, it exits Tor through a public Tor exit node. Anyone conducting monitoring or analysis will only see the traffic coming from the Tor exit node and will not be able to determine the original IP address of the request.

 

Figure 1: Malicious tactics and techniques aided by Tor, mapped to the MITRE ATT&CK framework

Malicious Tactics and Techniques Aided by Tor

Threat actors use Tor to create a layer of anonymity to conceal malicious activity at different stages of network compromise. Their tactics and techniques—illustrated in figure 1 above—include:

Pre-ATT&CK

  • Target Selection [TA0014]
  • Technical Information Gathering [TA0015]
    • Conduct Active Scanning [T1254]
    • Conduct Passive Scanning [T1253]
    • Determine domain and IP address space [T1250]
    • Identify security defensive capabilities [T1263]
  • Technical Weakness Identification [TA0018]

ATT&CK

Key Indicators of Malicious Activity via Tor

While Tor obfuscates a user from being identified through standard security tools, network defenders can leverage various network, endpoint, and security appliance logs to detect the use of Tor, including potentially malicious activity involving Tor, through indicator- or behavior-based analysis.

Using an indicator-based approach, network defenders can leverage security information and event management (SIEM) tools and other log analysis platforms to flag suspicious activities involving the IP addresses of Tor exit nodes. The list of Tor exit node IP addresses is actively maintained by the Tor Project’s Exit List Service, which offers both real-time query and bulk download interfaces (see https://blog.torproject.org/changes-tor-exit-list-service). Organizations preferring bulk download may consider automated data ingest solutions, given the highly dynamic nature of the Tor exit list, which is updated hourly. Network defenders should closely inspect evidence of substantial transactions with Tor exit nodes—revealed in netflow, packet capture (PCAP), and web server logs—to infer the context of the activity and to discern any malicious behavior that could represent reconnaissance, exploitation, C2, or data exfiltration.

Using a behavior-based approach, network defenders can uncover suspicious Tor activity by searching for the operational patterns of Tor client software and protocols. Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) ports commonly affiliated with Tor include 9001, 9030, 9040, 9050, 9051, and 9150. Highly structured Domain Name Service (DNS) queries for domain names ending with the suffix torproject.org is another behavior exhibited by hosts running Tor software. In addition, DNS queries for domains ending in .onion is a behavior exhibited by misconfigured Tor clients, which may be attempting to beacon to malicious Tor hidden services.

Organizations should research and enable the pre-existing Tor detection and mitigation capabilities within their existing endpoint and network security solutions, as these often employ effective detection logic. Solutions such as web application firewalls, router firewalls, and host/network intrusion detection systems may already provide some level of Tor detection capability.

Mitigations

Organizations can implement mitigations of varying complexity and restrictiveness to reduce the risk posed by threat actors who use Tor to carry out malicious activities. However, mitigation actions can also impact the access of legitimate users who leverage Tor to protect their privacy when visiting an organization’s internet-facing assets. Organizations should evaluate their probable risk, available resources, and impact to legitimate, non-malicious, Tor users before applying mitigation actions. 

  • Most restrictive approach: Block all web traffic to and from public Tor entry and exit nodes. Organizations that wish to take a conservative or less resource-intensive approach to reduce the risk posed by threat actors’ use of Tor should implement tools that restrict all traffic—malicious and legitimate—to and from Tor entry and exit nodes. Of note, blocking known Tor nodes does not completely eliminate the threat of malicious actors using Tor for anonymity, as additional Tor network access points, or bridges, are not all listed publicly. See table 1 for the most restrictive mitigation practices.

Table 1: Most restrictive mitigation practices

Type Level of Effort Technical Implementation

Impact 

Baseline Activity Low/Medium

Require organization to maintain up-to-date lists of known Tor exit and entry node IP addresses.

Public lists are available on the internet, but frequency of updates and accuracy varies depending on the source. The Tor Project maintains an authoritative list

Up-to-date awareness of known Tor nodes to enable blocking
External Policies Medium

Set external policies to block incoming traffic from known Tor exit nodes to prevent malicious reconnaissance and exploit attempts.

Network security tools (e.g., next-generation firewalls, proxies) may have configuration settings to apply these policies.

Block inbound network traffic, both malicious and legitimate, from reaching the organization’s domain from known Tor exit nodes
Internal Policies Medium

Set internal policies to block outgoing traffic to Tor entry nodes to prevent data exfiltration and C2 traffic.

Network security tools (e.g., next-generation firewalls, proxies) may have configuration settings to apply these policies.

Block outbound network traffic, both malicious and legitimate, from leaving the organization’s domain into known Tor entry nodes

 

  • Less restrictive approach: Tailor monitoring, analysis, and blocking of web traffic to and from public Tor entry and exit nodes. There are instances in which legitimate users may leverage Tor for internet browsing and other non-malicious purposes. For example, deployed military or other overseas voters may use Tor as part of the voting process to escape monitoring by foreign governments. Such users may use Tor when visiting elections-related websites, to check voter registration status, or to mark and then cast absentee ballots via email or web portal. Similarly, some users may use Tor to avoid tracking by advertisers when browsing the internet. Organizations that do not wish to block legitimate traffic to/from Tor entry/exit nodes should consider adopting practices that allow for network monitoring and traffic analysis for traffic from those nodes, and then consider appropriate blocking. This approach can be resource intensive but will allow greater flexibility and adaptation of defensive.

Table 2: Less restrictive mitigation practices

Type Level of Effort Technical Implementation Impact
Known Tor Nodes Low/Medium

Require the organization to maintain up-to-date lists of known Tor exit and entry node IP addresses.

The Tor Project maintains an authoritative list

Up-to-date awareness of known Tor nodes to enable baselining/allow blocking
SIEM Correlation Low/Medium Integrate network security and SIEM tools that correlate logs. Enhanced understanding of legitimate/expected Tor use for inbound/outbound traffic
Baseline Medium

Analyze traffic to determine normal patterns of behavior; legitimate vs. anomalous uses of Tor.

Baseline existing Tor traffic to/from known entry/exit nodes over a period of months.

Inspect traffic to understand legitimate traffic; level-set the organization’s risk tolerance for blocking or allowing Tor traffic to/from specific services.

Baseline understanding of legitimate vs. potentially anomalous Tor uses.
Internal / External Policies Medium/High

Institute behavioral signatures/rules to block unexpected/potentially malicious activity and allow legitimate activity.

Examine activity between any ephemeral port and Tor IP—this could be malicious data exfiltration or C2 traffic (except where use of outbound Tor entry nodes is expected).

Monitor for use of TCP/UDP ports 9001, 9030, 9040, 9050, 9051, 9150, and TCP ports 443* and 8443.

Monitor and/or block inbound connections from Tor exit nodes to IP addresses and ports for which external connections are not expected (i.e., other than VPN gateways, mail ports, web ports).

Associated ports are applicable for client -> guard/relay traffic monitoring and analysis but not monitoring for exit node -> a network destination.

Monitor and examine any large dataflows between networks and Tor IP addresses, regardless of port, as this could be unauthorized data exfiltration.

*Since port 443 is the most common port for secure web traffic, generically monitoring 443 may produce a high volume of false positives; network traffic tools can be used to assist in this analysis.

Legitimate traffic via Tor entry/exit nodes is permitted and unexpected/potentially malicious activity via Tor entry/exit nodes is blocked

 

  • Blended approach: Block all Tor traffic to some resources, allow and monitor for others. Given the various licit and illicit uses of Tor, a blended approach may be an appropriate risk mitigation strategy for some organizations (i.e., intentionally allowing traffic to/from Tor only for specific websites and services where legitimate use may be expected and blocking all Tor traffic to/from non-excepted processes/services). This may require continuous re-evaluation as an entity considers its own risk tolerance associated with different applications. The level of effort to implement this approach is high.

Considerations for Blocking Use of Tor

Sophisticated threat actors may leverage additional anonymization technologies—such as virtual private networks (VPNs)—and configurable features within Tor—such as Tor bridges and pluggable transports—to circumvent detection and blocking. Blocking the use of known Tor nodes may not effectively mitigate all hazards but may protect against less sophisticated actors. For example, blocking outbound traffic to known Tor entry nodes could have an appreciable impact in blocking less sophisticated malware from successfully beaconing out to hidden C2 machines obfuscated by Tor. Ultimately, each entity must consider its own internal thresholds and risk tolerance when determining a risk mitigation approach associated with Tor.

Contact Information

To report suspicious or criminal activity related to information found in this Joint Cybersecurity Advisory, contact your local FBI field office at www.fbi.gov/contact-us/field, or the FBI’s 24/7 Cyber Watch (CyWatch) at (855) 292-3937 or by e-mail at CyWatch@fbi.gov. When available, please include the following information regarding the incident: date, time, and location of the incident; type of activity; number of people affected; type of equipment used for the activity; the name of the submitting company or organization; and a designated point of contact. To request incident response resources or technical assistance related to these threats, contact CISA at CISAServiceDesk@cisa.dhs.gov.

Disclaimer

This document is marked TLP:WHITE. Disclosure is not limited. Sources may use TLP:WHITE when information carries minimal or no foreseeable risk of misuse, in accordance with applicable rules and procedures for public release. Subject to standard copyright rules, TLP:WHITE information may be distributed without restriction. For more information on the Traffic Light Protocol, see http://www.us-cert.gov/tlp/.

References

Revisions

  • July 1, 2020: Initial Version

This product is provided subject to this Notification and this Privacy & Use policy.

AA20-183A: Defending Against Malicious Cyber Activity Originating from Tor

This post was originally published on this site

Original release date: July 1, 2020

Summary

This advisory uses the MITRE Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK®) and Pre-ATT&CK framework. See the ATT&CK for Enterprise and Pre-ATT&CK frameworks for referenced threat actor techniques.

This advisory—written by the Cybersecurity Security and Infrastructure Security Agency (CISA) with contributions from the Federal Bureau of Investigation (FBI)—highlights risks associated with Tor, along with technical details and recommendations for mitigation. Cyber threat actors can use Tor software and network infrastructure for anonymity and obfuscation purposes to clandestinely conduct malicious cyber operations.[1],[2],[3]

Tor (aka The Onion Router) is software that allows users to browse the web anonymously by encrypting and routing requests through multiple relay layers or nodes. This software is maintained by the Tor Project, a nonprofit organization that provides internet anonymity and anti-censorship tools. While Tor can be used to promote democracy and free, anonymous use of the internet, it also provides an avenue for malicious actors to conceal their activity because identity and point of origin cannot be determined for a Tor software user. Using the Onion Routing Protocol, Tor software obfuscates a user’s identity from anyone seeking to monitor online activity (e.g., nation states, surveillance organizations, information security tools). This is possible because the online activity of someone using Tor software appears to originate from the Internet Protocol (IP) address of a Tor exit node, as opposed to the IP address of the user’s computer.

CISA and the FBI recommend that organizations assess their individual risk of compromise via Tor and take appropriate mitigations to block or closely monitor inbound and outbound traffic from known Tor nodes.

Click here for a PDF version of this report.

Risk Evaluation

Malicious cyber actors use Tor to mask their identity when engaging in malicious cyber activity impacting the confidentiality, integrity, and availability of an organization’s information systems and data. Examples of this activity include performing reconnaissance, penetrating systems, exfiltrating and manipulating data, and taking services offline through denial-of-service attacks and delivery of ransomware payloads. Threat actors have relayed their command and control (C2) server communications—used to control systems infected with malware—through Tor, obscuring the identity (location and ownership) of those servers.

The use of Tor in this context allows threat actors to remain anonymous, making it difficult for network defenders and authorities to perform system recovery and respond to cyberattacks. Organizations that do not take steps to block or monitor Tor traffic are at heightened risk of being targeted and exploited by threat actors hiding their identity and intentions using Tor.

The risk of being the target of malicious activity routed through Tor is unique to each organization. An organization should determine its individual risk by assessing the likelihood that a threat actor will target its systems or data and the probability of the threat actor’s success given current mitigations and controls. This assessment should consider legitimate reasons that non-malicious users may prefer to, or need to, use Tor for accessing the network. Organizations should evaluate their mitigation decisions against threats to their organization from advanced persistent threats (APTs), moderately sophisticated attackers, and low-skilled individual hackers, all of whom have leveraged Tor to carry out reconnaissance and attacks in the past.

Technical Details

Tor obfuscates the source and destination of a web request. This allows users to conceal information about their activities on the web—such as their location and network usage—from the recipients of that traffic, as well as third parties who may conduct network surveillance or traffic analysis. Tor encrypts a user’s traffic and routes the traffic through at least three Tor nodes, or relays, so that the user’s starting IP address and request is masked from network and traffic observers during transit. Once the request reaches its intended destination, it exits Tor through a public Tor exit node. Anyone conducting monitoring or analysis will only see the traffic coming from the Tor exit node and will not be able to determine the original IP address of the request.

 

Figure 1: Malicious tactics and techniques aided by Tor, mapped to the MITRE ATT&CK framework

Malicious Tactics and Techniques Aided by Tor

Threat actors use Tor to create a layer of anonymity to conceal malicious activity at different stages of network compromise. Their tactics and techniques—illustrated in figure 1 above—include:

Pre-ATT&CK

  • Target Selection [TA0014]
  • Technical Information Gathering [TA0015]
    • Conduct Active Scanning [T1254]
    • Conduct Passive Scanning [T1253]
    • Determine domain and IP address space [T1250]
    • Identify security defensive capabilities [T1263]
  • Technical Weakness Identification [TA0018]

ATT&CK

Key Indicators of Malicious Activity via Tor

While Tor obfuscates a user from being identified through standard security tools, network defenders can leverage various network, endpoint, and security appliance logs to detect the use of Tor, including potentially malicious activity involving Tor, through indicator- or behavior-based analysis.

Using an indicator-based approach, network defenders can leverage security information and event management (SIEM) tools and other log analysis platforms to flag suspicious activities involving the IP addresses of Tor exit nodes. The list of Tor exit node IP addresses is actively maintained by the Tor Project’s Exit List Service, which offers both real-time query and bulk download interfaces (see https://blog.torproject.org/changes-tor-exit-list-service). Organizations preferring bulk download may consider automated data ingest solutions, given the highly dynamic nature of the Tor exit list, which is updated hourly. Network defenders should closely inspect evidence of substantial transactions with Tor exit nodes—revealed in netflow, packet capture (PCAP), and web server logs—to infer the context of the activity and to discern any malicious behavior that could represent reconnaissance, exploitation, C2, or data exfiltration.

Using a behavior-based approach, network defenders can uncover suspicious Tor activity by searching for the operational patterns of Tor client software and protocols. Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) ports commonly affiliated with Tor include 9001, 9030, 9040, 9050, 9051, and 9150. Highly structured Domain Name Service (DNS) queries for domain names ending with the suffix torproject.org is another behavior exhibited by hosts running Tor software. In addition, DNS queries for domains ending in .onion is a behavior exhibited by misconfigured Tor clients, which may be attempting to beacon to malicious Tor hidden services.

Organizations should research and enable the pre-existing Tor detection and mitigation capabilities within their existing endpoint and network security solutions, as these often employ effective detection logic. Solutions such as web application firewalls, router firewalls, and host/network intrusion detection systems may already provide some level of Tor detection capability.

Mitigations

Organizations can implement mitigations of varying complexity and restrictiveness to reduce the risk posed by threat actors who use Tor to carry out malicious activities. However, mitigation actions can also impact the access of legitimate users who leverage Tor to protect their privacy when visiting an organization’s internet-facing assets. Organizations should evaluate their probable risk, available resources, and impact to legitimate, non-malicious, Tor users before applying mitigation actions. 

  • Most restrictive approach: Block all web traffic to and from public Tor entry and exit nodes. Organizations that wish to take a conservative or less resource-intensive approach to reduce the risk posed by threat actors’ use of Tor should implement tools that restrict all traffic—malicious and legitimate—to and from Tor entry and exit nodes. Of note, blocking known Tor nodes does not completely eliminate the threat of malicious actors using Tor for anonymity, as additional Tor network access points, or bridges, are not all listed publicly. See table 1 for the most restrictive mitigation practices.

Table 1: Most restrictive mitigation practices

Type Level of Effort Technical Implementation

Impact 

Baseline Activity Low/Medium

Require organization to maintain up-to-date lists of known Tor exit and entry node IP addresses.

Public lists are available on the internet, but frequency of updates and accuracy varies depending on the source. The Tor Project maintains an authoritative list

Up-to-date awareness of known Tor nodes to enable blocking
External Policies Medium

Set external policies to block incoming traffic from known Tor exit nodes to prevent malicious reconnaissance and exploit attempts.

Network security tools (e.g., next-generation firewalls, proxies) may have configuration settings to apply these policies.

Block inbound network traffic, both malicious and legitimate, from reaching the organization’s domain from known Tor exit nodes
Internal Policies Medium

Set internal policies to block outgoing traffic to Tor entry nodes to prevent data exfiltration and C2 traffic.

Network security tools (e.g., next-generation firewalls, proxies) may have configuration settings to apply these policies.

Block outbound network traffic, both malicious and legitimate, from leaving the organization’s domain into known Tor entry nodes

 

  • Less restrictive approach: Tailor monitoring, analysis, and blocking of web traffic to and from public Tor entry and exit nodes. There are instances in which legitimate users may leverage Tor for internet browsing and other non-malicious purposes. For example, deployed military or other overseas voters may use Tor as part of the voting process to escape monitoring by foreign governments. Such users may use Tor when visiting elections-related websites, to check voter registration status, or to mark and then cast absentee ballots via email or web portal. Similarly, some users may use Tor to avoid tracking by advertisers when browsing the internet. Organizations that do not wish to block legitimate traffic to/from Tor entry/exit nodes should consider adopting practices that allow for network monitoring and traffic analysis for traffic from those nodes, and then consider appropriate blocking. This approach can be resource intensive but will allow greater flexibility and adaptation of defensive.

Table 2: Less restrictive mitigation practices

Type Level of Effort Technical Implementation Impact
Known Tor Nodes Low/Medium

Require the organization to maintain up-to-date lists of known Tor exit and entry node IP addresses.

The Tor Project maintains an authoritative list

Up-to-date awareness of known Tor nodes to enable baselining/allow blocking
SIEM Correlation Low/Medium Integrate network security and SIEM tools that correlate logs. Enhanced understanding of legitimate/expected Tor use for inbound/outbound traffic
Baseline Medium

Analyze traffic to determine normal patterns of behavior; legitimate vs. anomalous uses of Tor.

Baseline existing Tor traffic to/from known entry/exit nodes over a period of months.

Inspect traffic to understand legitimate traffic; level-set the organization’s risk tolerance for blocking or allowing Tor traffic to/from specific services.

Baseline understanding of legitimate vs. potentially anomalous Tor uses.
Internal / External Policies Medium/High

Institute behavioral signatures/rules to block unexpected/potentially malicious activity and allow legitimate activity.

Examine activity between any ephemeral port and Tor IP—this could be malicious data exfiltration or C2 traffic (except where use of outbound Tor entry nodes is expected).

Monitor for use of TCP/UDP ports 9001, 9030, 9040, 9050, 9051, 9150, and TCP ports 443* and 8443.

Monitor and/or block inbound connections from Tor exit nodes to IP addresses and ports for which external connections are not expected (i.e., other than VPN gateways, mail ports, web ports).

Associated ports are applicable for client -> guard/relay traffic monitoring and analysis but not monitoring for exit node -> a network destination.

Monitor and examine any large dataflows between networks and Tor IP addresses, regardless of port, as this could be unauthorized data exfiltration.

*Since port 443 is the most common port for secure web traffic, generically monitoring 443 may produce a high volume of false positives; network traffic tools can be used to assist in this analysis.

Legitimate traffic via Tor entry/exit nodes is permitted and unexpected/potentially malicious activity via Tor entry/exit nodes is blocked

 

  • Blended approach: Block all Tor traffic to some resources, allow and monitor for others. Given the various licit and illicit uses of Tor, a blended approach may be an appropriate risk mitigation strategy for some organizations (i.e., intentionally allowing traffic to/from Tor only for specific websites and services where legitimate use may be expected and blocking all Tor traffic to/from non-excepted processes/services). This may require continuous re-evaluation as an entity considers its own risk tolerance associated with different applications. The level of effort to implement this approach is high.

Considerations for Blocking Use of Tor

Sophisticated threat actors may leverage additional anonymization technologies—such as virtual private networks (VPNs)—and configurable features within Tor—such as Tor bridges and pluggable transports—to circumvent detection and blocking. Blocking the use of known Tor nodes may not effectively mitigate all hazards but may protect against less sophisticated actors. For example, blocking outbound traffic to known Tor entry nodes could have an appreciable impact in blocking less sophisticated malware from successfully beaconing out to hidden C2 machines obfuscated by Tor. Ultimately, each entity must consider its own internal thresholds and risk tolerance when determining a risk mitigation approach associated with Tor.

Contact Information

To report suspicious or criminal activity related to information found in this Joint Cybersecurity Advisory, contact your local FBI field office at www.fbi.gov/contact-us/field, or the FBI’s 24/7 Cyber Watch (CyWatch) at (855) 292-3937 or by e-mail at CyWatch@fbi.gov. When available, please include the following information regarding the incident: date, time, and location of the incident; type of activity; number of people affected; type of equipment used for the activity; the name of the submitting company or organization; and a designated point of contact. To request incident response resources or technical assistance related to these threats, contact CISA at CISAServiceDesk@cisa.dhs.gov.

Disclaimer

This document is marked TLP:WHITE. Disclosure is not limited. Sources may use TLP:WHITE when information carries minimal or no foreseeable risk of misuse, in accordance with applicable rules and procedures for public release. Subject to standard copyright rules, TLP:WHITE information may be distributed without restriction. For more information on the Traffic Light Protocol, see http://www.us-cert.gov/tlp/.

References

Revisions

  • July 1, 2020: Initial Version

This product is provided subject to this Notification and this Privacy & Use policy.

System Requirements for VMware Products 2020

This post was originally published on this site

Most system (hardware) requirements on a single page. I hope it will be useful to you. I also have added the links to the source. vSphere vCenter Server Appliances 7.0 (VCSA) https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vcenter.install.doc/GUID-457EAE1F-B08A-4E64-8506-8A3FA84A0446.html Product Size vCPU vRAM Storage Comment vCenter VCSA tiny 2 12 GB 315 GB 10 hosts and 100 VMsdefault storage vCenter VCSA tiny-lstorageContinue reading “System Requirements for VMware Products 2020”

Removing Inaccessible Objects in VSAN

This post was originally published on this site

Reading Time: 2 minutes While building out a new vSAN Cluster, I received an error for the vSAN object health. I clicked the “Repair Objects Immediately” button, and it thought about it for a minute – but ultimately it was a wasted effort. The “Purge Inaccessible VM Swap Objects” button was up next. But unfortunately, like its partner, it…

Solution: Nested ESXi 6.7 install fails with the error ‘tryFormatDevice’

This post was originally published on this site

Nested ESXi deployments are one of the best ways to test VMware products and their functionality in a lab setup. While I was setting up my 6.7 P01 environment to do some testing, I ran into a weird issue of ‘tryFormatDevice’ that i hadn’t seen before with nested installs. The error is about the installer […]

The post Solution: Nested ESXi 6.7 install fails with the error ‘tryFormatDevice’ appeared first on vPirate.

VMware vSphere 7 what’s new

This post was originally published on this site

Reading Time: 5 minutes vSphere 7 is built for supporting modern applications and the hybrid cloud. In the coming years, enterprises will build more and more applications using cloud-native tools and methods. There is a lot more complexity in deploying and managing modern applications. vSphere 7 with Kubernetes (formerly known as Project Pacific) is based on VMware Cloud Foundation…

Announcing the Porting Assistant for .NET

This post was originally published on this site

.NET Core is the future of .NET! Version 4.8 of the .NET Framework is the last major version to be released, and Microsoft has stated it will receive only bug-, reliability-, and security-related fixes going forward. For applications where you want to continue to take advantage of future investments and innovations in the .NET platform, you need to consider porting your applications to .NET Core. Also, there are additional reasons to consider porting applications to .NET Core such as benefiting from innovation in Linux and open source, improved application scaling and performance, and reducing licensing spend. Porting can, however, entail significant manual effort, some of which is undifferentiated such as updating references to project dependencies.

When porting .NET Framework applications, developers need to search for compatible NuGet packages and update those package references in the application’s project files, which also need to be updated to the .NET Core project file format. Additionally, they need to discover replacement APIs since .NET Core contains a subset of the APIs available in the .NET Framework. As porting progresses, developers have to wade through long lists of compile errors and warnings to determine the best or highest priority places to continue chipping away at the task. Needless to say, this is challenging, and the added friction could be a deterrent for customers with large portfolios of applications.

Today we announced the Porting Assistant for .NET, a new tool that helps customers analyze and port their .NET Framework applications to .NET Core running on Linux. The Porting Assistant for .NET assesses both the application source code and the full tree of public API and NuGet package dependencies to identify those incompatible with .NET Core and guides developers to compatible replacements when available. The suggestion engine for API and package replacements is designed to improve over time as the assistant learns more about the usage patterns and frequency of missing packages and APIs.

The Porting Assistant for .NET differs from other tools in that it is able to assess the full tree of package dependencies, not just incompatible APIs. It also uses solution files as the starting point, which makes it easier to assess monolithic solutions containing large numbers of projects, instead of having to analyze and aggregate information on individual binaries. These and other abilities gives developers a jump start in the porting process.

Analyzing and porting an application
Getting started with porting applications using the Porting Assistant for .NET is easy, with just a couple of prerequisites. First, I need to install the .NET Core 3.1 SDK. Secondly I need a credential profile (compatible with the AWS Command Line Interface (CLI), although the CLI is not used or required). The credential profile is used to collect compatibility information on the public APIs and packages (from NuGet and core Microsoft packages) used in your application and public NuGet packages that it references. With those prerequisites taken care of, I download and run the installer for the assistant.

With the assistant installed, I check out my application source code and launch the Porting Assistant for .NET from the Start menu. If I’ve previously assessed some solutions, I can view and open those from the Assessed solutions screen, enabling me to pick up where I left off. Or I can select Get started, as I’ll do here, from the home page to begin assessing my application’s solution file.

I’m asked to select the credential profile I want to use, and here I can also elect to opt-in to share my telemetry data. Sharing this data helps to further improve on suggestion accuracy for all users as time goes on, and is useful in identifying issues, so we hope you consider opting-in.

I click Next, browse to select the solution file that I want, and then click Assess to begin the analysis. For this post I’m going to use the open source NopCommerce project.

When analysis is complete I am shown the overall results – the number of incompatible packages the application depends on, APIs it uses that are incompatible, and an overall Portability score. This score is an estimation of the effort required to port the application to .NET Core, based on the number of incompatible APIs it uses. If I’m working on porting multiple applications I can use this to identify and prioritize the applications I want to start on first.

Let’s dig into the assessment overview to see what was discovered. Clicking on the solution name takes me to a more detailed dashboard and here I can see the projects that make up the application in the solution file, and for each the numbers of incompatible package and API dependencies, along with the portability score for each particular project. The current port status of each project is also listed, if I’ve already begun porting the application and have reopened the assessment.

Note that with no project selected in the Projects tab the data shown in the Project references, NuGet packages, APIs, and Source files tabs is solution-wide, but I can scope the data if I wish by first selecting a project.

The Project references tab shows me a graphical view of the package dependencies and I can see where the majority of the dependencies are consumed, in this case the Npp.Core, Npp.Services, and Npp.Web.Framework projects. This view can help me decide where I might want to start first, so as to get the most ‘bang for my buck’ when I begin. I can also select projects to see the specific dependencies more clearly.

The NuGet packages tab gives me a look at the compatible and incompatible dependencies, and suggested replacements if available. The APIs tab lists the incompatible APIs, what package they are in, and how many times they are referenced. Source files lists all of the source files making up the projects in the application, with an indication of how many incompatible API calls can be found in each file. Selecting a source file opens a view showing me where the incompatible APIs are being used and suggested package versions to upgrade to, if they exist, to resolve the issue. If there is no suggested replacement by simply updating to a different package version then I need to crack open a source editor and update the code to use a different API or approach. Here I’m looking at the report for DependencyRegistrar.cs, that exists in the Nop.Web project, and uses the Autofac NuGet package.

Let’s start porting the application, starting with the Nop.Core project. First, I navigate back to the Projects tab, select the project, and then click Port project. During porting the tool will help me update project references to NuGet packages, and also updates the project files themselves to the newer .NET Core formats. I have the option of either making a copy of the application’s solution file, project files, and source files, or I can have the changes made in-place. Here I’ve elected to make a copy.

Clicking Save copies the application source code to the selected location, and opens the Port projects view where I can set the new target framework version (in this case netcoreapp3.1), and the list of NuGet dependencies for the project that I need to upgrade. For each incompatible package the Porting Assistant for .NET gives me a list of possible version upgrades and for each version, I am shown the number of incompatible APIs that will either remain, or will additionally become incompatible. For the package I selected here there’s no difference but for cases where later versions potentially increase the number of incompatible APIs that I would then need to manually fix up in the source code, this indication helps me to make a trade-off decision around whether to upgrade to the latest versions of a package, or stay with an older one.

Once I select a version, the Deprecated API calls field alongside the package will give me a reminder of what I need to fix up in a code editor. Clicking on the value summarizes the deprecated calls.

I continue with this process for each package dependency and when I’m ready, click Port to have the references updated. Using my IDE, I can then go into the source files and work on replacing the incompatible API calls using the Porting Assistant for .NET‘s source file and deprecated API list views as a reference, and follow a similar process for the other projects in my application.

Improving the suggestion engine
The suggestion engine behind the Porting Assistant for .NET is designed to learn and give improved results over time, as customers opt-in to sharing their telemetry. The data models behind the engine, which are the result of analyzing hundreds of thousands of unique packages with millions of package versions, are available on GitHub. We hope that you’ll consider helping improve accuracy and completeness of the results by contributing your data. The user guide gives more details on how the data is used.

The Porting Assistant for .NET is free to use and is available now.

— Steve

VMware Extends No. 1 Ranking in Cloud System and Service Management by Global Analyst Firm

This post was originally published on this site

VMware vRealize Cloud Management has achieved the number 1 slot in the fast-growing sector of cloud system and service management. According to the IDC report, “Worldwide Cloud System and Service Management Software Market Shares, 2019: SaaS and ITOM Drive Growth,” VMware’s share was more than 5 percentage points ahead of the No. 2 ranked vendor.

The post VMware Extends No. 1 Ranking in Cloud System and Service Management by Global Analyst Firm appeared first on VMware Cloud Management.

50 Conceptos de vRealize Operations Manager – Parte 1

This post was originally published on this site
50 Conceptos de vRealize Operations Manager - Parte 1

50 Conceptos vRealize Operations Manager

 

La idea de esta serie de Posts es obtener una idea global de los principales conceptos de vRealize Operations Manager, vROps para los amigos, con el fin de familiarizarnos con esta poderosa herramienta.

Es preocupante la cantidad de infraestructuras de VMware que disponen no solo de licencias de vROps, sino que además el servicio está instalado y funcionando aunque es muy bajo el porcetaje de empresas que realmente saben utilizarlo.
El motivo es muy claro y no es otro que la falta de conceptos sumado a muy poco tiempo invertido en la herramienta.

Comprender 50 conceptos clave es la diferencia entre leer y comprender un texto técnico.

Vamos a comenzar con una pequeña descripción de qué es vROps.

vRealize Operations Manager es un sistema de Gestión de la Capacidad y Monitorización de los Recursos que, a través de su sistema de auto-aprendizaje, tiene la capacidad de proveer recomendaciones y alertas de forma proactiva.

La instalación y configuración de vROps es muy pero muy simple ya que está compuesto por pocos componentes y tanto el despliegue como la configuración inicial se hacen en pocos minutos.

1-Nodo: Un nodo es una instancia del Appliance de vROps que cumple una determinada función. Existen 4 tipos de Nodos para dar servicio al Cluster de vRealize Operations Manager como son el Master, Replica, Data y el Collector.
Cualquier despliegue de vROps requiere, al menos, un nodo.
El instalador es el mismo para los Nodos del Master, Replica y Data y el OVA en versión 8.1 ocupa un tamaño de 2.57 GB.

vRealize Operations Manager

 

2-Cluster: Todo despliegue de vROps incluye un Cluster. No importa si nuestra instalación es muy simple y está compuesto por un único Nodo, siempre tendremos el servicio de Cluster. Tal servicio estará Online mientras esté operando y para ejecutar tareas de mantenimiento como añadir otro Nodo, Actualizar o Parchear debemos detener el servicio dejando el servicio de Cluster en estado Offline.

Nodo vRealize Operations Manager

TIP: Para acceder al panel de gestión del cluster apuntamos a https://fqdn-vrops/admin, mientas que para las tareas del día a día de vROps utilizamos https://fqdn-vrops/ui.

3-Master (Node): El Master Node es el servidor principal de vROps que se encarga de todas las tareas como recoger métricas, analizarlas, clasificarlas, almacenarlas, presentarlas, enviar notificaciones y otras tareas como autenticación y gestión de APIs y el interface de usuario.
El primer Nodo del Cluster ejecutará todas las tareas mencionadas anteriormente. En pruebas de concepto o instalaciones pequeñas que no requieran alta disponibilidad es posible utilizar un único Nodo en el Cluster que operará como Master.

vRealize Operations Master Node

TIP: Al tratarse de un sistema de monitorización y análisis es un MUST apuntar a un NTP, el mismo al que estén apuntando sistemas como DC’s, vCenter, nodos NSX, etc.

4-Replica (Node): El Replica Node o Master Replica Node nos permite disponer de Alta Disponibilidad en el Cluster de vROps.
Los Nodos Master y Replica se repartirán las tareas de Análisis y, en caso de fallo del Master Nodo, el Replica Node se encargará de la continuidad del servicio. Los datos del Master Node se replicarán en el Replica Node incrementando la disponibilidad del Cluster.
El despliegue en Alta Disponibilidad requiere de un servicio externo de balanceo que se encargue de la monitorización de los Nodos y de la distribución del tráfico.

TIP: Crea reglas de anti-afinidad para distribuir la ejecución de los Nodos del Cluster en Hosts diferentes. Idéntico caso si estamos utilizando Storage DRS.
Aplicar las recomendaciones del sizing recomendado por VMware según el tamaño y considerar aprovisionar los appliances (Nodos) sobre almacenamiento con buen rendimiento.

5-Data (Node): La forma de escalar nuestro Cluster de vROps después de haber desplegado un Master y un Replica Node es desplegar una o más instancias de Data Node.
El Data Node se encarga del almacenamiento de los Datos, especialmente orientado para grandes despliegues como indican los diseños validados de VMware.

TIP: Tenemos a nuestra disposición el vROpsSizer, una excelente web que nos permite introducir los datos de nuestro entorno y nos recomendará el número de Nodos y sus correspondientes capacidades.
Link: https://vropssizer.vmware.com/sizing-wizard/choose-installation

Sizer vRealize Operations Manager

https://vropssizer.vmware.com/sizing-wizard/choose-installation

 

6-Collector: Los collectors o recopiladores son instancias mínimas de vROps que únicamente recogen métricas y las reenvían al servicio de Cluster. Es especialmente útil cuando contamos con una infraestructura distribuida en varios Sites, incluso separados por miles de kilómetros, ya que nos permite disponer de un único punto de control para múltiples Sites.
Otro caso de uso de los Collectors es distribuir la carga (o liberar al Master y Replica Node) para la tarea de recepción de métricas.

7-Adapter o Account: Anteriormente llamado Adapter o recientemente renombrado a Account (o Cloud Account) es el nombre que utilizamos para llamar al conector que apunta a un determinado recurso o servicio para poder recibir métricas, parámetros y estados.
Por ejemplo el Account más común es el que utilizamos para conectarnos a una o múltiples instancias de vCenter Server.

TIP: Utilizar cuentas de servicio para conectar nuestro vROps al vCenter nos permite identificar las tareas y carga de trabajo que procede de vROps en nuestras instancias de vCenter.

vROps account

8-Metrica: Valor de un determinado contador asociado a una instancia de un objeto que proviene de un Adaptador o Cuenta Cloud.
Ejemplo: Valor [65] de un contador [CPU Workload] asociado a una instancia de un objeto [VM] que proviene de una Cuenta Cloud [vCenter].
Las cuentas Cloud/Adaptadores cuentan con múltiples tipos de objetos asociados y cada objeto tiene su colección de propiedades, estados y métricas.

TIP: Independientemente de conocer cómo funciona vROps, si no estamos familiarizados con ciertos valores de referencia como son las métricas principales será algo complicado comprender qué es lo que está ocurriendo con ese objeto de la infraestructura.

Metrica vRealize Operations Manager

9-Threshold: Existen determinados umbrales que delimitan el estado de las métricas de los objetos. vRealize Operations Manager utiliza 4 colores que son verde, amarillo, naranja y rojo. Siguiendo con el ejemplo del consumo de vCPU podemos decir que un valor de consumo de hasta 69% se mostrará en color verde. De 70 a 79% el color del estado será amarillo y resentará un aviso. Un valor de entre 80 y 89% estará “pintado” de color naranja y deberá ser considerado con una alerta y si el consumo de vCPU supera el 90% de uso veremos el estado del contador en color rojo y, en consecuencia, debemos considerar el estado como critico.
En el supuesto caso que no estemos recibiendo métricas del objeto el color que represente el estado será gris.

TIP: No todas las cargas de trabajo deberían ser consideradas bajo la vara del mismo umbral ya que no es lo mismo un Cluster crítico en Producción que otro destinado a pruebas y desarrollo. Podemos adaptar los Thresholds a través de las Políticas.

vRealize Operations Manager Threshold

vROps Threshold

 

10-Parametro: Es muy fácil confundir una métrica con un parámetro. Una métrica puede cambiar su valor con cierta frecuencia dependiendo de lo que esté representando.
Un parámetro nos sirve para obtener valores que cambian muy poco con el tiempo o que directamente no cambian. Como ejemplos podemos mencionar la versión del SO del guest, el nombre de una BBDD, la dirección IP de un controlador de dominio o el número de discos de capacidad de un Disk Group en vSAN.
Además es posible crear grupos de objetos dinámicos en base a propiedades como Tags o Etiquetas y gestionar esos objetos agrupados, por ejemplo, en base a un SLA en común.

TIP: Podemos definir nuestros Parametros utilizando el API de vROps y extender de esta forma la monitorización de determinadas aplicaciones sin necesidad de desarrollar con el SDK.

Parametros vRealize Operations Manager

 

Bonus Track -> eBook gratuito “Maximizing VMware vRealize Operations” por David Davis

eBook Gratuito vRealize Operations Manager

Link: https://blogs.vmware.com/management/2018/10/free-ebook-the-gorilla-guide-to-vmware-vrealize-operations-by-vexpert-david-m-davis.html

 

Como decía al principio, espero que les resulte de utilidad. Si te gustó por favor ayudame a compartirlo para que pueda llegar a mas gente. 

Nos vemos en el próximo Post!!!

 

Rolling back a version of ESXi

This post was originally published on this site
There is an option in VMware where after you have performed an major upgrade of ESXi you can roll back to your previous version. The benefit of this is that you would not need to reinstall your ESXi and its configuration if you had issues with the new software. I had to do this on one occassion in my lab where I upgraded from 6.5 to 6.7 and my VMs would not run because the CPU was not supported in 6.7. Please remember if you are using ISO method to upgrade ESXi please ensure you select “Upgrade ESXi, preserve VMFS datastore”. Selecting “Install ESXi, preserve VMFS datastore” does not mean preserving datastore means retaining ESXi as it will still do a clean install of ESXi. This method does not work for vSphere 7.0 as there are changes to the partitions on the boot device.

Below are the steps to roll back to a previous version which is quite straight forward. As always perform an backup of your host configuration before you upgrade or rollback (KB2042141). I have tested roll back where I have performed the upgrade via update manager, CLI upgrade and via ISO.

How do you access the recovery menu?
You may have noticed this screen when you boot up your ESXi host

On your keyboard press “Shift + R” which will take you to the recovery menu before the automatic boot counter times out.
As you can see on this recovery menu we have nothing to roll back as I have not updated the host as yet.


Now go through your chosen method of upgrade for your host and then next time at boot up menu go into the recovery menu (Shift + R) you should see a screen similar like below

You can see the default hypervisor is set as 6.7 and the available ESXi version we can roll back to is 6.5. So press Y” to confirm you wish to roll back and remember it will not prompt again to confirm so make sure that you wish to roll back.
ESXi will reboot and you will be back to your previous installed version.
Its a nice feature for a quick rollback if you had issues with upgrades but don’t leave it too long after an upgrade for rollbacks.