Introducing Data Expedition And MTP/IP
百度 4.在“学校”这里的学校就指学生平日中学习成长的地方。Data Expedition makes MTP/IP. If you’re a network engineer used to seeing TCP/IP or UDP/IP, then MTP/IP would catch your attention. MTP stands for “multipurpose transaction protocol.” MTP was built to overcome the algorithmic challenges TCP has filling pipes.
TCP doesn’t cope?as efficiently as it might with latency, congestion, and packet loss. MTP was designed as a transport that can run at?maximum, pipe-filling efficiency without human intervention no matter what the link bandwidth and conditions are between two MTP endpoints. In that sense, you might think that MTP sounds an awful lot like WAN optimization, but it really isn’t.
MTP/IP Is?Not WAN Optimization
WAN optimization isn’t primarily about filling pipes. WAN op is?more about reducing the number of bits on the wire using tokenization, deduplication, and compression and eliminating round trips via local acknowledgments and application proxies. Data Expedition’s MTP/IP is about figuring out just how much data can be sent through a link, and sending just that much–but no more.
MTP/IP isn’t actually a new type of IP protocol, like TCP (IP protocol 6) or UDP (IP protocol 17) or even OSPF (IP protocol 89).?Rather, MTP/IP is UDP with flow control, acknowledgements, etc. embedded at the application layer, rather than the transport?layer. That means that?the network part of the communications stack doesn’t have to worry about packet delivery. The Data Expedition application using the proprietary MTP/IP worries about it.
Use Cases For MTP/IP
Many?Data Expedition?customers leverage MTP/IP using ExpeDat, a “drop in replacement for conventional FTP applications.” In this model, customers license an ExpeDat server for a certain number of simultaneous connections, and are able to distribute the client freely both within the company and to partners who might have need of the accelerated transfer service.
Use cases for?Data Expedition include moving large data files over wide area networks including the Internet?(Disney, Amagi, Harris, Children’s Hospital of Philadelphia), and predictable access to data over unpredictable infrastructure (Associated Press, LexisNexis).
Data Expedition Announces CloudDat
On April 24, 2017, Data Expedition announced CloudDat, a new use-case for MTP/IP aimed at organizations needing to move data into and out of the public cloud. The application for CloudDat is straightforward. Moving large data sets from your local data center into the public cloud for processing is time-consuming. CloudDat aims to minimize the time taken to get that data into the cloud by?filling the available bandwidth between your network and the public cloud network.
CloudDat is available on AWS, GCP, Azure, and Oracle Cloud.
On AWS, CloudDat is available from the AWS Marketplace, and can be consumed in hourly increments for?$1.50, plus the hourly cost of the instance you’re running CloudDat on. GCP, Azure, and Oracle Cloud don’t currently support hourly metered charges, but Data Expedition hopes to support such functionality on those clouds should it become available.
Data Expedition also notes that MTP/IP is built directly into the Oracle DIVA media asset management system. This means that Oracle DIVA customers will benefit from MTP/IP efficiency without having to run?CloudDat separately.
The View From The Hot Aisle
Data Expedition fills an interesting niche, going after well-known challenges with TCP so many of us are familiar with. TCP goes back decades, designed for networking at a time when the global Internet as it exists today was not yet conceived of. Many efforts over the years, both commercial and in standards bodies, have addressed?the sticky problem of making TCP performance both efficient and consistent in the light of unpredictable path characteristics.
Thus, MTP/IP is an interesting technology that should be helpful to organizations trying to make the most of their Internet transport or expensive leased WAN lines. TCP isn’t especially good at that. Perhaps MTP/IP is.
MTP/IP is also interesting in its deployment model–a client application you load on supported operating systems, including Windows, Linux, macOS, FreeBSD, AIX and more. This means you can put MTP/IP where it’s needed, without the architectural concerns introduced by WAN optimization appliances.?The?app should scale well on these operating systems, as the app can be threaded across multiple cores.
In addition, MTP/IP might even be complementary to your WAN optimization solution. For example, Data Expedition shared that some Silver Peak customers were seeing 2x-4x performance improvement when running MTP/IP’s accelerated stream through the Silver Peak platform.?Of course, that implies an MTP/IP data stream with an uncompressed and unencrypted payload, per normal for WAN optimization traffic flows.
There might even be an IoT use case for MTP/IP. The software libraries comprising?MTP/IP fit into kilobytes, and not megabytes. Therefore, it’s conceivable that an ultralightweight unikernel OS created specifically as an IoT platform could include MTP/IP, offering data transfer efficiency from the IoT device to wherever the data needs to be processed.
I think the public cloud play is a completely appropriate use-case for MTP/IP as used by?CloudDat. If I had a frequent?need to move big files between my site and the public cloud, I’d be interested to give CloudDat a try. Saving time when transferring large amounts of data across the wide area is a big deal.
If I were doing a POC of any enhanced data mover?product such as what Data Expedition offers, I’d be checking at least the following.
- Quantify bang for the buck. Performance improvement of MTP/IP based transfer vs. FTP (or other transfer method) + WAN optimization. I’m assuming you’ve got WAN optimization already. I’m also assuming you’ve integrated your WAN optimization into your public cloud presence.
- Go deep on actual costs. Pricing when spinning up CloudDat in various public clouds. What is the metering model? MTP/IP will use CPU to do what it does, so does that impact the ultimate cost to you when consuming the service via cloud? In AWS, I don’t believe that it does if I understood Data Expedition correctly, but verify the exact pricing model in your specific circumstance.
- Create a failure situation. Find out how MTP/IP recovers during a total WAN outage. What happens when the file transfer is interrupted for minutes, hours, or longer? Auto-recovery from failure scenarios is a must-have.
- Understand automation options. How well can ExpeDat and CloudDat tools be automated, and how complex is the automation tooling? I’ve worked with some optimized file transfer packages before that yes, indeed, offered automation, but were so obtuse as to be prohibitive to use.
