How to mock new Builder with some new objects as parameters
default RestTemplate restTemplate(String baseUrl, Optional<Map<String, String>> headers) {
return new RestTemplateBuilder()
.requestFactory(
() -> new BufferingClientHttpRequestFactory(new SimpleClientHttpRequestFactory()))
.uriTemplateHandler(new DefaultUriBuilderFactory(baseUrl))
.interceptors(new RequestResponseLoggingInterceptor(headers))
.errorHandler(new CustomResponseErrorHandler())
.build();
}
I tried spying, but its not working
RestTemplateService restTemplateService = Mockito.spy(RestTemplateService.class);
RestTemplate restTemplate = Mockito.mock(RestTemplate.class);
when(restTemplateService.restTemplate(anyString(), any())).thenReturn(restTemplate);
I am trying to call below API from my SpringBoot Application:
"https://api.coindesk.com/v1/bpi/currentprice.json"
Since this API is returning javaScript instead of Json is am getting error as :
no suitable HttpMessageConverter found for response type [class ] and content type [application/javascript].
kindly advise how resolve this issue. Below is my code:
package org.digital.cur.service;
public class DigitalCurService
{
public static ResponseEntity digitalCurService1()
{
final HttpHeaders headers= setHttpHeaders();
RestTemplate restTemplate = new RestTemplate();
HttpEntity<String> request= new HttpEntity<>(headers);
ResponseEntity<Currentprice> responseEntity = restTemplate.exchange("https://api.coindesk.com/v1/bpi/currentprice.json", HttpMethod.GET, request, Currentprice.class );
return responseEntity;
}
private static HttpHeaders setHttpHeaders()
{
HttpHeaders headers = new HttpHeaders();
MediaType mt = new MediaType("application", "javascript");
headers.setContentType(mt);
return headers;
}
}
So I have written a simple Azure Function (AF) that accepts (via Http Post method) an IFormCollection, loops through the file collection, pushes each file into an Azure Blob storage container and returns the url to each file.
The function itself works perfectly when I do a single file or multiple file post through Postman using the 'multipart/form-data' header. However when I try to post a file through an xUnit test, I get the following error:
System.IO.InvalidDataException : Multipart body length limit 16384 exceeded.
I have searched high and low for a solution, tried different things, namely;
Replicating the request object to be as close as possible to Postmans request.
Playing around with the 'boundary' in the header.
Setting 'RequestFormLimits' on the function.
None of these have helped so far.
The details are the project are as follows:
Azure Function v3: targeting .netcoreapp3.1
Startup.cs
public class Startup : FunctionsStartup
{
public IConfiguration Configuration { get; private set; }
public override void Configure(IFunctionsHostBuilder builder)
{
var x = builder;
InitializeConfiguration(builder);
builder.Services.AddSingleton(Configuration.Get<UploadImagesAppSettings>());
builder.Services.AddLogging();
builder.Services.AddSingleton<IBlobService,BlobService>();
}
private void InitializeConfiguration(IFunctionsHostBuilder builder)
{
var executionContextOptions = builder
.Services
.BuildServiceProvider()
.GetService<IOptions<ExecutionContextOptions>>()
.Value;
Configuration = new ConfigurationBuilder()
.SetBasePath(executionContextOptions.AppDirectory)
.AddJsonFile("appsettings.json")
.AddJsonFile("appsettings.Development.json", optional: true)
.AddEnvironmentVariables()
.Build();
}
}
UploadImages.cs
public class UploadImages
{
private readonly IBlobService BlobService;
public UploadImages(IBlobService blobService)
{
BlobService = blobService;
}
[FunctionName("UploadImages")]
[RequestFormLimits(ValueLengthLimit = int.MaxValue,
MultipartBodyLengthLimit = 60000000, ValueCountLimit = 10)]
public async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = "images")] HttpRequest req)
{
List<Uri> returnUris = new List<Uri>();
if (req.ContentLength == 0)
{
string badResponseMessage = $"Request has no content";
return new BadRequestObjectResult(badResponseMessage);
}
if (req.ContentType.Contains("multipart/form-data") && req.Form.Files.Count > 0)
{
foreach (var file in req.Form.Files)
{
if (!file.IsValidImage())
{
string badResponseMessage = $"{file.FileName} is not a valid/accepted Image file";
return new BadRequestObjectResult(badResponseMessage);
}
var uri = await BlobService.CreateBlobAsync(file);
if (uri == null)
{
return new ObjectResult($"Could not blob the file {file.FileName}.");
}
returnUris.Add(uri);
}
}
if (!returnUris.Any())
{
return new NoContentResult();
}
return new OkObjectResult(returnUris);
}
}
Exception Thrown:
The below exception is thrown at the second if statement above, when it tries to process req.Form.Files.Count > 0, i.e.
if (req.ContentType.Contains("multipart/form-data") && req.Form.Files.Count > 0) {}
Message:
System.IO.InvalidDataException : Multipart body length limit 16384 exceeded.
Stack Trace:
MultipartReaderStream.UpdatePosition(Int32 read)
MultipartReaderStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)
MultipartReader.ReadNextSectionAsync(CancellationToken cancellationToken)
FormFeature.InnerReadFormAsync(CancellationToken cancellationToken)
FormFeature.ReadForm()
DefaultHttpRequest.get_Form()
UploadImages.Run(HttpRequest req) line 42
UploadImagesTests.HttpTrigger_ShouldReturnListOfUploadedUris(String fileNames)
xUnit Test Project: targeting .netcoreapp3.1
Over to the xUnit Test project, basically I am trying to write an integration test. The project references the AF project and has the following classes:
TestHost.cs
public class TestHost
{
public TestHost()
{
var startup = new TestStartup();
var host = new HostBuilder()
.ConfigureWebJobs(startup.Configure)
.ConfigureServices(ReplaceTestOverrides)
.Build();
ServiceProvider = host.Services;
}
public IServiceProvider ServiceProvider { get; }
private void ReplaceTestOverrides(IServiceCollection services)
{
// services.Replace(new ServiceDescriptor(typeof(ServiceToReplace), testImplementation));
}
private class TestStartup : Startup
{
public override void Configure(IFunctionsHostBuilder builder)
{
SetExecutionContextOptions(builder);
base.Configure(builder);
}
private static void SetExecutionContextOptions(IFunctionsHostBuilder builder)
{
builder.Services.Configure<ExecutionContextOptions>(o => o.AppDirectory = Directory.GetCurrentDirectory());
}
}
}
TestCollection.cs
[CollectionDefinition(Name)]
public class TestCollection : ICollectionFixture<TestHost>
{
public const string Name = nameof(TestCollection);
}
HttpRequestFactory.cs: To create Http Post Request
public static class HttpRequestFactory
{
public static DefaultHttpRequest Create(string method, string contentType, Stream body)
{
var request = new DefaultHttpRequest(new DefaultHttpContext());
var contentTypeWithBoundary = new MediaTypeHeaderValue(contentType)
{
Boundary = $"----------------------------{DateTime.Now.Ticks.ToString("x")}"
};
var boundary = MultipartRequestHelper.GetBoundary(
contentTypeWithBoundary, (int)body.Length);
request.Method = method;
request.Headers.Add("Cache-Control", "no-cache");
request.Headers.Add("Content-Type", contentType);
request.ContentType = $"{contentType}; boundary={boundary}";
request.ContentLength = body.Length;
request.Body = body;
return request;
}
private static string GetBoundary(MediaTypeHeaderValue contentType, int lengthLimit)
{
var boundary = HeaderUtilities.RemoveQuotes(contentType.Boundary);
if (string.IsNullOrWhiteSpace(boundary.Value))
{
throw new InvalidDataException("Missing content-type boundary.");
}
if (boundary.Length > lengthLimit)
{
throw new InvalidDataException(
$"Multipart boundary length limit {lengthLimit} exceeded.");
}
return boundary.Value;
}
}
The MultipartRequestHelper.cs class is available here
And Finally the Test class:
[Collection(TestCollection.Name)]
public class UploadImagesTests
{
readonly UploadImages UploadImagesFunction;
public UploadImagesTests(TestHost testHost)
{
UploadImagesFunction = new UploadImages(testHost.ServiceProvider.GetRequiredService<IBlobService>());
}
[Theory]
[InlineData("testfile2.jpg")]
public async void HttpTrigger_ShouldReturnListOfUploadedUris(string fileNames)
{
var formFile = GetFormFile(fileNames);
var fileStream = formFile.OpenReadStream();
var request = HttpRequestFactory.Create("POST", "multipart/form-data", fileStream);
var response = (OkObjectResult)await UploadImagesFunction.Run(request);
//fileStream.Close();
Assert.True(response.StatusCode == StatusCodes.Status200OK);
}
private static IFormFile GetFormFile(string fileName)
{
string fileExtension = fileName.Substring(fileName.IndexOf('.') + 1);
string fileNameandPath = GetFilePathWithName(fileName);
IFormFile formFile;
var stream = File.OpenRead(fileNameandPath);
switch (fileExtension)
{
case "jpg":
formFile = new FormFile(stream, 0, stream.Length,
fileName.Substring(0, fileName.IndexOf('.')),
fileName)
{
Headers = new HeaderDictionary(),
ContentType = "image/jpeg"
};
break;
case "png":
formFile = new FormFile(stream, 0, stream.Length,
fileName.Substring(0, fileName.IndexOf('.')),
fileName)
{
Headers = new HeaderDictionary(),
ContentType = "image/png"
};
break;
case "pdf":
formFile = new FormFile(stream, 0, stream.Length,
fileName.Substring(0, fileName.IndexOf('.')),
fileName)
{
Headers = new HeaderDictionary(),
ContentType = "application/pdf"
};
break;
default:
formFile = null;
break;
}
return formFile;
}
private static string GetFilePathWithName(string filename)
{
var outputFolder = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location);
return $"{outputFolder.Substring(0, outputFolder.IndexOf("bin"))}testfiles\\{filename}";
}
}
The test seems to be hitting the function and req.ContentLength does have a value. Considering this, could it have something to do with the way the File Streams are being managed? Perhaps not the right way?
Any inputs on this would be greatly appreciated.
Thanks
UPDATE 1
As per this post, I have also tried setting the ValueLengthLimit and MultipartBodyLengthLimit in the Startup of the Azure Function and/or the Test Project as opposed to attributes on the Azure Function. The exception then changed to:
"The inner stream position has changed unexpectedly"
Following this, I then set the fileStream position in the test project to SeekOrigin.Begin. I started getting the same error:
"Multipart body length limit 16384 exceeded."
It took me a 50km bike ride and a good nights sleep but I finally figured this one out :-).
The Azure function (AF) accepts an HttpRequest object as a parameter with the name of 'req' i.e.
public async Task Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = "images")] HttpRequest req)
The hierarchy of the files object in the HttpRequest object (along with the parameter names) is as follows:
HttpRequest -> req
FormCollection -> Form
FormFileCollection -> Files
This is what the AF accepts and one would access the files collection by using req.Form.Files
In my test case, instead of posting a FormCollection object, I was trying to post a Stream of a file to the Azure Function.
var formFile = GetFormFile(fileNames);
var fileStream = formFile.OpenReadStream();
var request = HttpRequestFactory.Create("POST", "multipart/form-data", fileStream);
As a result of this, req.Form had a Stream value that it could not interpret and the req.Form.Files was raising an exception.
In order to rectify this, I had to do the following:
Revert all changes made as part of UPDATE 1. This means that I removed the 'RequestFormLimits' settings from the Startup file and left them as attributes on the functions Run method.
Instantiate a FormFileCollection object and add the IFormFile to it
Instantiate a FormCollection object using this FormFileCollection as a parameter.
Add the FormCollection to the request object.
To achieve the above, I had to make the following changes in code.
Change 'Create' method in the HttpRequestFactory
public static DefaultHttpRequest Create(string method, string contentType, FormCollection formCollection)
{
var request = new DefaultHttpRequest(new DefaultHttpContext());
var boundary = $"----------------------------{DateTime.Now.Ticks.ToString("x")}";
request.Method = method;
request.Headers.Add("Cache-Control", "no-cache");
request.Headers.Add("Content-Type", contentType);
request.ContentType = $"{contentType}; boundary={boundary}";
request.Form = formCollection;
return request;
}
Add a private static GetFormFiles() method
I wrote an additional GetFormFiles() method that calls the existing GetFormFile() method, instantiate a FormFileCollection object and add the IFormFile to it. This method in turn returns a FormFileCollection.
private static FormFileCollection GetFormFiles(string fileNames)
{
var formFileCollection = new FormFileCollection();
foreach (var file in fileNames.Split(','))
{
formFileCollection.Add(GetFormFile(file));
}
return formFileCollection;
}
Change the Testmethod
The test method calls the GetFormFiles() to get a FormFileCollection then
instantiates a FormCollection object using this FormFileCollection as a parameter and then passes the FormCollection object as a parameter to the HttpRequest object instead of passing a Stream.
[Theory]
[InlineData("testfile2.jpg")]
public async void HttpTrigger_ShouldReturnListOfUploadedUris(string fileNames)
{
var formFiles = GetFormFiles(fileNames);
var formCollection = new FormCollection(null, formFiles);
var request = HttpRequestFactory.Create("POST", "multipart/form-data", formCollection);
var response = (OkObjectResult) await UploadImagesFunction.Run(request);
Assert.True(response.StatusCode == StatusCodes.Status200OK);
}
So in the end the issue was not really with the 'RequestFormLimits' but rather with the type of data I was submitting in the POST message.
I hope this answer provides a different perspective to someone that comes across the same error message.
Cheers.
Can I have some help here on the below issue:
Calling the transformer to transform the input object to Map Object and calling handler, the handler is missing header values added before.
Why on transforming payload to Map object losing all headers?
//Adding header here setHeader("t", "t");
#ResponseBody
public EmResponse getAuditTrail(#Valid #RequestBody NGAuditTrailEntry auditEntry) {
LOG.info("Audit Service Called, creating new audit " + auditEntry);
AuditCreationFlow.CreateAuditGateway auditGateway = applicationContext.getBean(AuditCreationFlow.CreateAuditGateway.class);
MessageBuilder messageBuilder = MessageBuilder.withPayload(auditEntry).setHeader("t", "t");
Object response = auditGateway.createAudit(messageBuilder.build());
EmResponse res = new EmResponse();
LOG.info("Done with Audit creation. Response " + response);
return res;
}
//Integration flow starts here
public IntegrationFlow createAuditGatewayFlow() {
LOG.debug("Entered to spring integration flow to create the Audit entry");
return IntegrationFlows.from("auditInputChannel")
.handle(auditObjTransformer, "transformToEjbCompatible")
.handle(ejbCaller, "callEjb")
.get();
}
//Transforming payload object to map
#Component
public class AuditObjTransformer {
private final Logger LOG = LoggerFactory.getLogger(this.getClass());
#Transformer
public Object transformToEjbCompatible(NGAuditTrailEntry ngAuditTrailEntry, Map<String, Object> headers) {
LOG.debug("Transforming the NGAuditTrailEntry To AuditEntry object which is EJB compatible");
//#TODO - Tranformation code goes here.
String s = ngAuditTrailEntry.getObjectName();
Map<String, String> m = new HashMap<>();
m.put("x", s);
return m;
}
//Here in this handler, not getting headers what I added in the rest service above.
public class EJBCaller {
private final Logger LOG = LoggerFactory.getLogger(this.getClass());
public Object callEjb(Object payload, Map<String, Object> headers) throws EJBResponseException {
LOG.debug("Calling Audit EJB to crated Audit entry.");
//#TODO EJB calling code goese here.
LOG.debug("Returned from EJB after creating Audit entry. Returned value" + payload);
return payload;
}
If the transform is other than map then no issues in headers.
Thanks,
Siva
callEjb(Object payload, Map<String, Object> headers)
If payload is a Map, you have that payload in the payload and the headers method arguments at the same time.
To make it working and carry exactly headers to that Map argument you should use #Headers annotation on it:
* Annotation which indicates that a method parameter should be bound to the headers of a
* message. The annotated parameter must be assignable to {#link java.util.Map} with
* String keys and Object values.
in previous si versions (si 2.11 to be specific and spring 3.1.1) getStandardRequestHeaderNames could be overrided to include Additional Application specific objects in the si message header. Our application relied on this ability (may be wrongfully so) to override this method and supply a custom POJO to be carried downstream consisting of many splitters, aggregators etc. The app used an ws inbound gateway and used the header-mapper attribute to specify the custom soap header mapper.
Any clues on the reasoning behind why getStandardRequestHeaderNames cannot be overriden?
Need some advise on how I can migrate this to the current spring release.
The requirement is to extract elements from soapHeader and map them to an SI message headers as an POJO and send it down stream.
All help appreciated.
Code Snippet: Works with older versions of spring
<int-ws:inbound-gateway id="webservice-inbound-gateway"
request-channel="input-request-channel"
reply-channel="output-response-channel"
header-mapper="CustomSoapHeaderMapper"
marshaller="marshaller"
unmarshaller="marshaller" />
#Component("CustomSoapHeaderMapper")
public class CustomSoapHeaderMapper extends DefaultSoapHeaderMapper {
private static final Logger logger = Logger.getLogger("CustomSoapHeaderMapper");
public static final String HEADER_SEARCH_METADATA = SearchMetadata.HEADER_ATTRIBUTE_NAME;
public static final String HEADER_SERVICE_AUDIT = "XXXXXXXX";
// Use simulation if security token is set to this value
public static final String SECURITY_TOKEN_SIMULATION = "XXXX";
private static final List<String> CUSTOM_HEADER_NAMES = new ArrayList<String>();
static {
CUSTOM_HEADER_NAMES.add(WebServiceHeaders.SOAP_ACTION);
CUSTOM_HEADER_NAMES.add(HEADER_SEARCH_METADATA);
}
private int version =SearchMetadata.VERSION_CURRENT;
public void setVersion(int version) {
this.version = version;
}
#Override
protected List<String> getStandardRequestHeaderNames() {
return CUSTOM_HEADER_NAMES;
}
#Override
protected Map<String, Object> extractUserDefinedHeaders(SoapMessage source) {
// logger.log(Level.INFO,"extractUserDefinedHeaders");
// call base class to extract header
Map<String, Object> map = super.extractUserDefinedHeaders(source);
Document doc = source.getDocument();
SearchMetadata searchMetadata = new SearchMetadata();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try {
source.writeTo(baos);
baos.flush();
searchMetadata.setRequestXML(baos.toString());
baos.close();
} catch (IOException e1) {
}
//logger.log(Level.WARNING, "Incoming Message " + baos.toString());
SOAPMessage soapMessage = ((SaajSoapMessage) source).getSaajMessage();
// generate TransactionID with UUID value
String transactionID = UUID.randomUUID().toString();
// logger.log(Level.WARNING, "TransactionID=" + transactionID);
Date now = new Date();
searchMetadata.setTransactionID(transactionID);
searchMetadata.setRequestType(SearchMetadata.REQUEST_TYPE_SYNCHRONOUS);
searchMetadata.setRequestTime(now);// initialize the request time
searchMetadata.setReceivedTime(now);// mark time system receives request
searchMetadata.setVersion(version);
Map<String, Object> finalHeaders = new HashMap<String, Object>();
finalHeaders.put(HEADER_SEARCH_METADATA, searchMetadata);
if (!CollectionUtils.isEmpty(map)) {
// copy from other map
finalHeaders.putAll(map);
// check if ServiceAudit is available
SoapHeaderElement serviceAuditElement = null;
for (String key : map.keySet()) {
// logger.log(Level.WARNING, "SoapHeader.{0}", key);
if (StringUtils.contains(key, HEADER_SERVICE_AUDIT)) {
serviceAuditElement = (SoapHeaderElement) map.get(key);
break;
}
}
}
return finalHeaders;
}
// GK Key Thing here for performance improvement is avoiding marshalling
public gov.dhs.ice.ess.schema.ServiceAudit ExtractAuditHeader(Document doc) {
....
}
return serviceAudit;
}
}
Share, please, some code how would you like to see that.
Maybe you can just implement your own SoapHeaderMapper and inject it into WS Inbound Gateway?
You can still reuse your logic and copy/paste the standard behavior from the DefaultSoapHeaderMapper.
UPDATE
The test-case to demonstrate how to add user-defined header manually:
#Test
public void testCustomSoapHeaderMapper() {
DefaultSoapHeaderMapper mapper = new DefaultSoapHeaderMapper() {
#Override
protected Map<String, Object> extractUserDefinedHeaders(SoapMessage source) {
Map<String, Object> headers = super.extractUserDefinedHeaders(source);
headers.put("foo", "bar");
return headers;
}
};
mapper.setRequestHeaderNames("*");
SoapMessage soapMessage = mock(SoapMessage.class);
Map<String, Object> headers = mapper.toHeadersFromRequest(soapMessage);
assertTrue(headers.containsKey("foo"));
assertEquals("bar", headers.get("foo"));
}